Beamr's AWS S3 integration requires an initial setup involving the creation of an IAM role and an SNS topic, either via a provided CloudFormation template or manual configuration. The IAM role (BeamrCloudAccessRole) grants Beamr access to S3 buckets, while the SNS topic (BeamrS3EventTopic) handles S3 event notifications. After initial setup, for the identified source and destination buckets, users must configure permissions for Beamr to read from source buckets and write into destination buckets. For live monitoring workflows, S3 event notifications must be set up to alert Beamr of new file uploads. The integration supports one AWS account and region, and changing these details may disrupt existing workflows.

Provisioning a dedicated IAM profile allows the owner of the S3 instance to grant Beamr Cloud access to AWS resources without sharing AWS security credentials. This approach helps maintain permission boundaries by controlling access to specific AWS folders and actions permitted by Beamr Cloud.

We recommend granting only the necessary permissions and avoiding the use of AmazonS3AllAccess whenever possible.

This below instructions will walk you through the initial configuration required to integrate your AWS S3 storage with the Beamr platform.

1. Update your AWS account and region on Beamr

1. In the Integrations page of the Beamr web app (Left Menu), click on the 'Connect AWS' button.
2. In the "AWS Account Details" window that appears, enter your AWS Account ID, Select your Region and click 'Update'.

2. Create Beamr IAM role & SNS topic 

This step involves configuring AWS IAM roles and an SNS topic to handle notifications and permissions.

Beamr provides a CloudFormation template to automate this process, but you also have the option to configure everything manually. This section covers both methods.

Manually Create IAM role & SNS topic using AWS console

If you prefer not to use the CloudFormation template, follow these steps to configure the IAM role and SNS topic manually.

Step 1: Create the IAM Role

  1. Log in to AWS Console.
  2. Go to IAM Service:
    • In the AWS Management Console, navigate to IAM by typing "IAM" in the search bar.
  3. Create a New Role:
    • Go to Roles and click Create Role.
    • Select Another AWS account as the trusted entity, and enter Beamr's AWS account ID: 897367468997.
  4. Set Permissions:
    • Add the following permissions to the role:
      • Source Bucket Permissions: List and get objects (s3:ListBucket, s3:GetObject) for your source buckets.
      • Destination Bucket Permissions: Upload and set ACL for objects (s3:PutObject, s3:PutObjectAcl) for your destination buckets.
      • List All Buckets: List all S3 buckets (s3:ListAllMyBuckets)
  5. Create the Role:
    • Name the role BeamrCloudAccessRole.

Step 2: Create the SNS Topic

  1. Go to SNS Service:
    • In the AWS Management Console, navigate to SNS by typing "SNS" in the search bar.
  2. Create a New Topic:
    • Click Create topic, choose Standard, and name it BeamrS3EventTopic.
  3. Set Topic Policy:
    • Edit the topic's access policy to allow Beamr's AWS account to subscribe and manage the topic. Here is an example policy:
{
  "Version": "2008-10-17",
  "Id": "__default_policy_ID",
  "Statement": [
    {
      "Sid": "__default_statement_ID",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": [
        "SNS:Publish",
        "SNS:RemovePermission",
        "SNS:SetTopicAttributes",
        "SNS:DeleteTopic",
        "SNS:ListSubscriptionsByTopic",
        "SNS:GetTopicAttributes",
        "SNS:AddPermission",
        "SNS:Subscribe"
      ],
      "Resource": "arn:aws:sns:your-region:your-account-id:BeamrS3EventTopic",
      "Condition": {
        "StringEquals": {
          "AWS:SourceAccount": "your-account-id"
        }
      }
    },
    {
      "Sid": "__console_sub_0",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::897367468997:root"
      },
      "Action": "SNS:Subscribe",
      "Resource": "arn:aws:sns:your-region:your-account-id:BeamrS3EventTopic"
    }
  ]
}
Once these manual steps are completed, your initial setup will be in place. You can then proceed to configure the specific S3 buckets as needed.

Create IAM role & SNS topic using cloudFormation

Beamr provides a CloudFormation template that automates the process of creating an IAM role and an SNS topic required for the integration. The template sets up the following resources:
  1. IAM Role: BeamrCloudAccessRole – This role allows Beamr to access your S3 buckets with the necessary permissions.
  2. SNS Topic: BeamrS3EventTopic – This is used to handle S3 event notifications, such as when new files are added or updated in the bucket.

Please review and copy cloudFormation from here: 

AWSTemplateFormatVersion: '2010-09-09'
Description: One-time setup for Beamr integration with pre-defined SNS topic policy.

Resources:
  BeamrCloudAccessRole:
    Type: 'AWS::IAM::Role'
    Properties:
      RoleName: BeamrCloudAccessRole
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              AWS:
                - arn:aws:iam::897367468997:role/prod_bvcloud_storage_process_cluster-role
                - arn:aws:iam::897367468997:role/prod-beamr-cloud-backend-role
                - arn:aws:iam::897367468997:role/BVCloudWorkers
            Action: 'sts:AssumeRole'

      Policies:
        - PolicyName: BeamrS3AccessPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              # Input bucket permissions (placeholders)
              - Effect: Allow
                Action:
                  - 's3:ListBucket'
                  - 's3:GetObject'
                Resource:
                  - "arn:aws:s3:::PLACEHOLDER_INPUT_BUCKET"
                  - "arn:aws:s3:::PLACEHOLDER_INPUT_BUCKET/*"

              # Output bucket permissions (placeholders)
              - Effect: Allow
                Action:
                  - 's3:PutObject'
                  - 's3:PutObjectAcl'
                Resource:
                  - "arn:aws:s3:::PLACEHOLDER_OUTPUT_BUCKET"
                  - "arn:aws:s3:::PLACEHOLDER_OUTPUT_BUCKET/*"

              # Permission to list all buckets
              - Effect: Allow
                Action:
                  - 's3:ListAllMyBuckets'
                Resource: "*"

  BeamrS3EventTopic:
    Type: 'AWS::SNS::Topic'
    Properties:
      TopicName: BeamrS3EventTopic

  BeamrS3EventTopicPolicy:
    Type: 'AWS::SNS::TopicPolicy'
    Properties:
      Topics:
        - !Ref BeamrS3EventTopic
      PolicyDocument:
        Version: '2008-10-17'
        Id: "__default_policy_ID"
        Statement:
          - Sid: "__default_statement_ID"
            Effect: Allow
            Principal:
              AWS: "*"
            Action:
              - "SNS:Publish"
              - "SNS:RemovePermission"
              - "SNS:SetTopicAttributes"
              - "SNS:DeleteTopic"
              - "SNS:ListSubscriptionsByTopic"
              - "SNS:GetTopicAttributes"
              - "SNS:AddPermission"
              - "SNS:Subscribe"
            Resource: !Ref BeamrS3EventTopic
            Condition:
              StringEquals:
                AWS:SourceAccount: !Ref AWS::AccountId

          - Sid: "__console_sub_0"
            Effect: Allow
            Principal:
              AWS: "arn:aws:iam::897367468997:root"
            Action: "SNS:Subscribe"
            Resource: !Ref BeamrS3EventTopic

Outputs:
  BeamrRoleArn:
    Description: ARN of the role that Beamr will assume
    Value: !GetAtt BeamrCloudAccessRole.Arn

  SNSTopicArn:
    Description: ARN of the SNS topic for S3 event notifications
    Value: !Ref BeamrS3EventTopic

Explanation of CloudFormation Template

The CloudFormation template provisions the following components:
  1. IAM Role (BeamrCloudAccessRole):
    • Purpose: This role grants Beamr the necessary permissions to access your S3 buckets for video optimization.
    • Permissions:
      • It allows Beamr to assume the role using predefined roles (prod_bvcloud_storage_process_cluster-role, prod-beamr-cloud-backend-role, and BVCloudWorkers).
      • It has placeholder permissions for both input and output buckets:
        • Source Buckets: Permission to list and get objects (s3:ListBucket, s3:GetObject).
        • Destination Buckets: Permission to upload objects and set object ACLs (s3:PutObject, s3:PutObjectAcl).
        • It also grants permission to list all the S3 buckets in your account (s3:ListAllMyBuckets) to allow easier selection on Beamr cloud UI
  2. SNS Topic (BeamrS3EventTopic):
    • Purpose: This topic is used to handle S3 event notifications, such as new file uploads.
    • The topic allows Beamr's AWS account (897367468997) to subscribe to the notifications, and it provides general permissions for SNS actions, including publishing and subscribing.
  3. SNS Topic Policy (BeamrS3EventTopicPolicy):
    • Purpose: Defines access control for the SNS topic, ensuring that Beamr and your account can use it correctly.
 

3. Setup of source & destination buckets

After the initial setup is completed you would continue to configure the buckets you wish Beamr to read from or write into.
Once you identified your relevant buckets you can manually configure them by granting Beamr access to read from a source bucket and write to a destination bucket. Additionally, for source buckets on which you wish Beamr to preform a "Live Monitoring" workflow, you will need to configure an event notifications to trigger upon new uploads.
 

Set the Bucket Policy to Allow the Beamr IAM Role to Access the Buckets

To allow the IAM role created in the first step (BeamrCloudAccessRole) to access your S3 buckets, you will need to update the policy for both the source bucket and destination bucket.

Steps:

  1. Log in to AWS Console.
  2. Go to the S3 Service:
    • In the AWS Management Console, navigate to S3 by typing "S3" in the search bar.
  3. Open the Source or Destination Bucket:
    • Navigate to the bucket where you want to grant Beamr access.
  4. Go to Permissions Tab:
    • Click on the Permissions tab.
  5. Edit Bucket Policy:
    • Scroll down to Bucket policy and click Edit.
    • Add the following JSON policy to allow the Beamr IAM role to access the bucket:

For source bucket: 

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<your-account-id>:role/BeamrCloudAccessRole"
      },
      "Action": [
        "s3:GetObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::your-source-bucket",
        "arn:aws:s3:::your-source-bucket/*"
      ]
    }
  ]
}
For the destination bucket, modify the permissions to allow Beamr to write files to the bucket:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::<your-account-id>:role/BeamrCloudAccessRole"
      },
      "Action": [
        "s3:PutObject",
        "s3:PutObjectAcl"
      ],
      "Resource": [
        "arn:aws:s3:::your-destination-bucket",
        "arn:aws:s3:::your-destination-bucket/*"
      ]
    }
  ]
}
  • Replace your-source-bucket and your-destination-bucket with the actual bucket names.
  • Replace your-account-id with your AWS account ID.
Once you've saved the changes, the Beamr role will have the necessary permissions to access the source and destination buckets.

Update the Resource Section of the Beamr IAM Role

Next, you need to update the IAM role's Resource section to enable access to the specific buckets that Beamr will operate on. The policy already exists, but you need to replace the placeholders (PLACEHOLDER_INPUT_BUCKET, PLACEHOLDER_OUTPUT_BUCKET) with the actual bucket names.

Steps:

  1. Log in to AWS Console. Try this quick link or navigate manually following the next steps 
  2. Go to the IAM Service:
    • In the AWS Management Console, navigate to IAM by typing "IAM" in the search bar.
  3. Locate the BeamrCloudAccessRole:
    • Under Roles, search for BeamrCloudAccessRole.
  4. Edit the Policy:
    • In the policy attached to the role, look for BeamrS3AccessPolicy.
    • Click Edit policy and navigate to the JSON editor.
    • Replace the placeholders in the policy with the actual bucket names for the source and destination buckets. Example: 
      {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::your-source-bucket",
        "arn:aws:s3:::your-source-bucket/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:PutObjectAcl"
      ],
      "Resource": [
        "arn:aws:s3:::your-destination-bucket",
        "arn:aws:s3:::your-destination-bucket/*"
      ]
    }
  ]
}
​
      Replace your-source-bucket and your-destination-bucket with the actual bucket names.
  5. Save the policy 
This ensures that the Beamr IAM role has the correct permissions to access the specific source and destination buckets for your workflow.

Configure Event Notifications for "Live Monitoring" Workflows

If your workflow involves live monitoring, where Beamr needs to be notified whenever new files are uploaded to the source bucket, you need to configure S3 event notifications to send these events to the SNS topic created earlier (BeamrS3EventTopic).

Steps:

  1. Go to the S3 Service:
    • In the AWS Management Console, navigate to S3 and open your source bucket.
  2. Go to Properties Tab:
    • Click on the Properties tab.
  3. Scroll to Event Notifications:
    • Under Event notifications, click Create event notification.
  4. Configure Event Notification:
    • Name the event (e.g., BeamrLiveMonitoring).
    • Events: Choose the event types you want to monitor. Typically, you want to choose All object create events to trigger notifications when new objects are uploaded to the bucket.
    • Destination: Select SNS Topic and choose BeamrS3EventTopic.
  5. Save the Event Notification.
Now, any new uploads to the configured source bucket will trigger an event notification to the SNS topic, which will notify Beamr to start processing the files.

Note

Currently Beamr platform supports connectivity to one AWS account and region. In case you wish to change your existing AWS Account ID or Region,  do it with caution as it can invalidate your existing workflows!