Beamr's AWS S3 integration requires an initial setup involving the creation of an IAM role and an SNS topic, either via a provided CloudFormation template or manual configuration. The IAM role (BeamrCloudAccessRole) grants Beamr access to S3 buckets, while the SNS topic (BeamrS3EventTopic) handles S3 event notifications. After initial setup, for the identified source and destination buckets, users must configure permissions for Beamr to read from source buckets and write into destination buckets. For live monitoring workflows, S3 event notifications must be set up to alert Beamr of new file uploads. The integration supports one AWS account and region, and changing these details may disrupt existing workflows.

Provisioning a dedicated IAM profile allows the owner of the S3 instance to grant Beamr Cloud access to AWS resources without sharing AWS security credentials. This approach helps maintain permission boundaries by controlling access to specific AWS folders and actions permitted by Beamr Cloud.

We recommend granting only the necessary permissions and avoiding the use of AmazonS3AllAccess whenever possible.

This below instructions will walk you through the initial configuration required to integrate your AWS S3 storage with the Beamr platform.

1. Update your AWS account and region on Beamr

1. In the Integrations page of the Beamr web app (Left Menu), click on the 'Connect AWS' button.
2. In the "AWS Account Details" window that appears, enter your AWS Account ID, Select your Region and click 'Update'.

Beamr is currently deployed in the following regions:

  • US East (us-east-1)
  • Europe Central (eu-central-1)

If your selected region is not included in this list, your files will be processed in our default region, US East (us-east-1), which may result in longer processing times. If you would like Beamr to support your preferred region, please contact us for assistance.

 

2. Create Beamr IAM role & SNS topic 

This step involves configuring AWS IAM roles and an SNS topic to handle notifications and permissions.

Beamr provides a CloudFormation template to automate this process, but you also have the option to configure everything manually. This section covers both methods.

Create IAM role & SNS topic 

Step 2.1: Create the IAM Role

  1. Create a New Role - select Another AWS account as the trusted entity, and enter Beamr's AWS account ID: 124355658833 with the following principals : 
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "AWS": [
                        "arn:aws:iam::124355658833:role/prod-beamr-cloud-backend-role",
                        "arn:aws:iam::124355658833:role/BVCloudWorkers",
                        "arn:aws:iam::124355658833:role/prod_bvcloud_storage_process_cluster-role"
                    ]
                },
                "Action": "sts:AssumeRole"
            }
        ]
    }
  2. Name the role BeamrCloudAccessRole
  3. Set Permissions:
    • Add the following permissions to the role:
      • s3:ListBucket, s3:GetObject for your source buckets
      • s3:PutObject, s3:PutObjectAcl for your destination buckets.
      • s3:ListAllMyBuckets
      • Example:
        {
        	"Version": "2012-10-17",
        	"Statement": [
        		{
        			"Action": [
        				"s3:ListBucket",
        				"s3:GetObject"
        			],
        			"Resource": [
        				"arn:aws:s3:::your-source-bucket",
        				"arn:aws:s3:::your-source-bucket/*"
        			],
        			"Effect": "Allow"
        		},
        		{
        			"Action": [
        				"s3:PutObject",
        				"s3:PutObjectAcl"
        			],
        			"Resource": [
        				"arn:aws:s3:::your-destination-bucket",
        				"arn:aws:s3:::your-destination-bucket/*"
        			],
        			"Effect": "Allow"
        		},
        		{
        			"Action": [
        				"s3:ListAllMyBuckets"
        			],
        			"Resource": "*",
        			"Effect": "Allow"
        		}
        	]
        }
        

Step 2.2: Create the SNS Topic

*This step is relevant for workflows defined for live monitoring. If you are planning a batch workflow, please skip this step.

Name the topic as BeamrS3EventTopic then edit the topic's access policy to allow Beamr's AWS account to subscribe and manage the topic. Here is an example policy:

{
  "Version": "2008-10-17",
  "Id": "__default_policy_ID",
  "Statement": [
    {
      "Sid": "__default_statement_ID",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": [
        "SNS:Publish",
        "SNS:RemovePermission",
        "SNS:SetTopicAttributes",
        "SNS:DeleteTopic",
        "SNS:ListSubscriptionsByTopic",
        "SNS:GetTopicAttributes",
        "SNS:AddPermission",
        "SNS:Subscribe"
      ],
      "Resource": "arn:aws:sns:your-region:your-account-id:BeamrS3EventTopic",
      "Condition": {
        "StringEquals": {
          "AWS:SourceAccount": "your-account-id"
        }
      }
    },
    {
      "Sid": "__console_sub_0",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::124355658833:root"
      },
      "Action": "SNS:Subscribe",
      "Resource": "arn:aws:sns:your-region:your-account-id:BeamrS3EventTopic"
    }
  ]
}
You can then proceed to configure the specific S3 buckets as needed.

3. Setup of source & destination buckets

After the initial setup is completed you would continue to configure the buckets you wish Beamr to read from or write into.
Once you identified your relevant buckets you can manually configure them by granting Beamr access to read from a source bucket and write to a destination bucket. Additionally, for source buckets on which you wish Beamr to preform a "Live Monitoring" workflow, you will need to configure an event notifications to trigger upon new uploads.

Set the Bucket Policy to Allow the Beamr IAM Role to Access the Buckets

To allow the IAM role created in the first step (BeamrCloudAccessRole) to access your S3 buckets, you will need to update the policy for both the source bucket and destination bucket.

 Edit Bucket Policies, for source bucket: 

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<your-account-id>:role/BeamrCloudAccessRole"
      },
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::your-source-bucket",
        "arn:aws:s3:::your-source-bucket/*"
      ]
    }
  ]
}
For the destination bucket, modify the permissions to allow Beamr to write files to the bucket:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<your-account-id>:role/BeamrCloudAccessRole"
      },
      "Action": [
        "s3:PutObject",
        "s3:PutObjectAcl"
      ],
      "Resource": [
        "arn:aws:s3:::your-destination-bucket",
        "arn:aws:s3:::your-destination-bucket/*"
      ]
    }
  ]
}
  • Replace your-source-bucket and your-destination-bucket with the actual bucket names.
  • Replace your-account-id with your AWS account ID.
Once you've saved the changes, the Beamr role will have the necessary permissions to access the source and destination buckets.

4. Configure Event Notifications for "Live Monitoring" Workflows

For live monitoring where Beamr must be notified upon new file uploads, configure S3 bucket event notifications to send events to the SNS topic (BeamrS3EventTopic). In the AWS Management Console, open your source bucket, go to the Properties tab, and scroll to Event Notifications. Create a new event notification (e.g., BeamrLiveMonitoring), select "All object create events" as the trigger, and set the destination to the BeamrS3EventTopic. Once saved, any new uploads will automatically notify Beamr to process the files.