Tony Duong
Let's design a Netflix-like application using AWS Elemental MediaConvert and Ruby on Rails

Photo by Venti Views on Unsplash

Let's design a Netflix-like application using AWS Elemental MediaConvert and Ruby on Rails

In this article, we will learn how to build a scalable video streaming application using AWS Elemental MediaConvert and Ruby on Rails.

Tony Duong's photo
Tony Duong
·

13 min read

Table of contents

In this day and age, everyone is using video streaming platforms whether it is YouTube, Netflix or Amazon Prime video, all allowing you to watch videos in high quality. In this article, we will learn how to build a scalable video streaming application using AWS Elemental MediaConvert and Ruby on Rails.

Building a big application like Netflix is certainly not an easy task so we will focus only on building a video streaming application that allows users to upload videos and watch them in different qualities, while providing a secure, scalable and performant solution.

Why is converting raw video files important?

Of course, it is absolutely possible to store and stream raw video files (for example, mp4 video files), but it is not the most efficient way to do so. HLS (HTTP Live Streaming) for example is a video streaming protocol that has been developed by Apple and is widely used by video streaming platforms. HLS has several advantages over streaming raw video:

  • Adaptive Bitrate Streaming: HLS allows you to stream video files in different qualities, automatically switching to the best quality depending on the user's internet connection speed.

  • Segmented Video Files: HLS breaks down video files into small segments, allowing users to start watching the video before the entire video file is downloaded.

  • Efficient bandwidth usage: HLS delivers video files in small chunks, reducing the bandwidth usage and providing a better user experience.

Of course, there are other video streaming protocols such as MPEG-DASH, or WebRTC, each with their own advantages. In this article, we will focus on HLS.

Architecture of the application

AWS Elemental MediaConvert

AWS Elemental MediaConvert is a file-based video transcoding service with broadcast-grade features. It allows you to easily create video-on-demand (VOD) content for broadcast and multiscreen delivery at scale.

The service combines advanced video and audio capabilities with a simple web services interface and pay-as-you-go pricing.

Ruby on Rails

We will use Ruby on Rails to build both the backend and frontend of the web application. Ruby on Rails is a web application framework written in Ruby. We will also execute background jobs for converting the video files using SolidQueue, a simple and efficient background processing library for Ruby on Rails.

Amazon S3 coupled with Amazon CloudFront

Amazon S3 is an object storage service that offers industry-leading scalability, data availability, security, and performance. We will store in Amazon S3, static assets such as the raw video files uploaded by the users and converted video files by AWS Elemental MediaConvert.

Then we will use Amazon CloudFront to deliver the video files to the users by caching the content at edge locations. Users will be only be able to access the video files through CloudFront URLs authenticated by signed cookies.

Diagram of the architecture

As you can see in the diagram above, the flow of the application is as follows:

  1. The administator uploads a video file to Amazon S3 directly through Rails ActiveStorage.

  2. The form is submitted to the Rails backend.

  3. The Rails backend saves the video information (title, raw video S3 object key etc.) in the database and triggers a background job to convert the video file using AWS Elemental MediaConvert.

  4. In the background, AWS Elemental MediaConvert converts the raw video file into HLS video files.

  5. Periodically, the Rails backend checks the status of the conversion job and updates the database.

  6. After the conversion is completed, AWS Elemental MediaConvert stores the final HLS video files in Amazon S3.

  7. The viewers are now able to watch the video by first getting the Cloudfront URL and setting the signed cookies.

  8. The viewers finally accesses the video URL through Amazon CloudFront with signed cookies.

In the next sections, we will go through the steps to build this application.

Step-by-step guide

Upload the video file to Amazon S3 via Rails ActiveStorage

First, we need to set up Rails ActiveStorage to upload the video files to Amazon S3.

# config/storage.yml
amazon_s3_assets: # bucket for storing raw video files
  service: S3
  access_key_id: <%= ENV["AWS_ACCESS_KEY_ID"] %>
  secret_access_key: <%= ENV["AWS_SECRET_ACCESS_KEY"] %>
  region: ap-northeast-1
  bucket: tonystrawberry-netflix-assets-<%= Rails.env %>

amazon_s3_output_assets: # bucket for storing converted video files
  service: S3
  access_key_id: <%= ENV["AWS_ACCESS_KEY_ID"] %>
  secret_access_key: <%= ENV["AWS_SECRET_ACCESS_KEY"] %>
  region: ap-northeast-1
  bucket: tonystrawberry-netflix-output-videos-<%= Rails.env %>

We will directly upload the video from the client side to Amazon S3 using Rails ActiveStorage using the built-in direct_upload method. For more information, please refer to the official guide.

# app/views/movies/_form.html.erb

<%= form_with model: movie, url: edit ? editor_update_movie_path(movie) : editor_create_movie_path do |f| %>
  ... 
  <%= f.file_field :video, direct_upload: true %>
  <%= f.submit I18n.t("views.editor.movies.new.save") %>
<% end %>

Upon submitting the form, the video file will be uploaded to Amazon S3 and the form will be submitted to the Rails backend.

Trigger a background job to convert the video file using AWS Elemental MediaConvert

Below is the controller code for handling the form submission, saving the movie data and triggering a background job to convert the video file using AWS Elemental MediaConvert.

# app/controllers/editor/movies_controller.rb
def create
  @movie = Movie.new(movie_params)
  @movie.save

  ConvertVideoToHlsFormatJob.perform_now(movie_id: id) # trigger the background job
end

ConvertVideoToHlsFormatJob is the background job that will update the conversion status in the database and trigger the conversion of the video file using AWS Elemental MediaConvert API.

# app/jobs/convert_video_to_hls_format_job.rb
class ConvertVideoToHlsFormatJob < ApplicationJob
  queue_as :default

  def perform(movie_id:)
    movie = Movie.find(movie_id)

    media_convert_job_id = MediaConvert::Executor.new.execute(
      source_s3_key: "#{Rails.application.config.active_storage.service_configurations["amazon_s3_assets"]["bucket"]}/#{movie.video.key}"
    )[:job_id]

    movie.update!(media_convert_job_id:, media_convert_status: :submitted)

    CheckHlsVideosJob.set(wait: 15.seconds).perform_later(movie_id:)
  end
end

The class MediaConvert::Executor contains all the logic to interact with the AWS Elemental MediaConvert API and set the conversion settings. As you can see from the content #job_settings method, there are lots of settings that can be configured such as the video resolution, bitrate, audio settings etc.

# app/services/media_convert/executor.rb
class MediaConvert::Base
  attr_reader :client

  def initialize
    client = Aws::MediaConvert::Client.new(
      region: ENV["AWS_REGION"],
      access_key_id: ENV["AWS_ACCESS_KEY_ID"],
      secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"]
    )

    endpoints = client.describe_endpoints

    @client = Aws::MediaConvert::Client.new(
      region: ENV["AWS_REGION"],
      access_key_id: ENV["AWS_ACCESS_KEY_ID"],
      secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
      endpoint: endpoints.endpoints[0].url
    )
  end
end

# app/services/media_convert/executor.rb
class MediaConvert::Executor < MediaConvert::Base
  attr_reader :response, :job_id

  def initialize
    super

    @response = nil
  end

  def execute(source_s3_key:)
    destination_s3_bucket_name = ENV["MEDIA_CONVERT_OUTPUT_BUCKET"]
    destination_video_s3_key = "#{source_s3_key}/HLS/video"
    destination_thumbnail_s3_key = "#{source_s3_key}/Thumbnails/thumbnail"

    settings = MediaConvert::Executor.job_settings(source_s3_key:, destination_s3_bucket_name:, destination_video_s3_key:, destination_thumbnail_s3_key:)

    @response = client.create_job(role: ENV["MEDIA_CONVERT_ROLE_ARN"], user_metadata: { "source_s3_key" => source_s3_key }, settings:)
    @job_id = @response.job.id

    {
      job_id: @job_id
    }
  end

  def self.job_settings(source_s3_key:, destination_s3_bucket_name:, destination_video_s3_key:, destination_thumbnail_s3_key:)
    {
      "timecode_config": {
        "source": "ZEROBASED"
      },
      "output_groups": [
        {
          "name": "Apple HLS",
          "outputs": [
            {
              "container_settings": {
                "container": "M3U8"
              },
              "video_description": {
                "width": 480,
                "height": 270,
                "codec_settings": {
                  "codec": "H_264",
                  "h264_settings": {
                    "max_bitrate": 1500000,
                    "rate_control_mode": "QVBR",
                    "scene_change_detect": "TRANSITION_DETECTION"
                  }
                }
              },
              "audio_descriptions": [
                {
                  "audio_source_name": "Audio Selector 1",
                  "codec_settings": {
                    "codec": "AAC",
                    "aac_settings": {
                      "bitrate": 96000,
                      "coding_mode": "CODING_MODE_2_0",
                      "sample_rate": 48000
                    }
                  }
                }
              ],
              "output_settings": {
                "hls_settings": {}
              },
              "name_modifier": "-480x270"
            },
            {
              "container_settings": {
                "container": "M3U8"
              },
              "video_description": {
                "width": 640,
                "height": 360,
                "codec_settings": {
                  "codec": "H_264",
                  "h264_settings": {
                    "max_bitrate": 2000000,
                    "rate_control_mode": "QVBR",
                    "scene_change_detect": "TRANSITION_DETECTION"
                  }
                }
              },
              "audio_descriptions": [
                {
                  "codec_settings": {
                    "codec": "AAC",
                    "aac_settings": {
                      "bitrate": 96000,
                      "coding_mode": "CODING_MODE_2_0",
                      "sample_rate": 48000
                    }
                  }
                }
              ],
              "output_settings": {
                "hls_settings": {}
              },
              "name_modifier": "-640x360"
            },
            {
              "container_settings": {
                "container": "M3U8"
              },
              "video_description": {
                "width": 1280,
                "height": 720,
                "codec_settings": {
                  "codec": "H_264",
                  "h264_settings": {
                    "max_bitrate": 4000000,
                    "rate_control_mode": "QVBR",
                    "scene_change_detect": "TRANSITION_DETECTION"
                  }
                }
              },
              "audio_descriptions": [
                {
                  "codec_settings": {
                    "codec": "AAC",
                    "aac_settings": {
                      "bitrate": 96000,
                      "coding_mode": "CODING_MODE_2_0",
                      "sample_rate": 48000
                    }
                  }
                }
              ],
              "output_settings": {
                "hls_settings": {}
              },
              "name_modifier": "-1280x720"
            },
            {
              "container_settings": {
                "container": "M3U8"
              },
              "video_description": {
                "width": 1920,
                "height": 1080,
                "codec_settings": {
                  "codec": "H_264",
                  "h264_settings": {
                    "max_bitrate": 8000000,
                    "rate_control_mode": "QVBR",
                    "scene_change_detect": "TRANSITION_DETECTION"
                  }
                }
              },
              "audio_descriptions": [
                {
                  "codec_settings": {
                    "codec": "AAC",
                    "aac_settings": {
                      "bitrate": 96000,
                      "coding_mode": "CODING_MODE_2_0",
                      "sample_rate": 48000
                    }
                  }
                }
              ],
              "output_settings": {
                "hls_settings": {}
              },
              "name_modifier": "-1920x1080"
            }
          ],
          "output_group_settings": {
            "type": "HLS_GROUP_SETTINGS",
            "hls_group_settings": {
              "segment_length": 10,
              "caption_language_setting": "OMIT",
              "destination": "s3://#{destination_s3_bucket_name}/#{destination_video_s3_key}",
              "min_segment_length": 0
            }
          }
        },
        {
          "name": "File Group",
          "outputs": [
            {
              "container_settings": {
                "container": "RAW"
              },
              "video_description": {
                "codec_settings": {
                  "codec": "FRAME_CAPTURE",
                  "frame_capture_settings": {
                    "framerate_numerator": 1,
                    "framerate_denominator": 3,
                    "max_captures": 100
                  }
                }
              }
            }
          ],
          "output_group_settings": {
            "type": "FILE_GROUP_SETTINGS",
            "file_group_settings": {
              "destination": "s3://#{destination_s3_bucket_name}/#{destination_thumbnail_s3_key}"
            }
          }
        }
      ],
      "follow_source": 1,
      "inputs": [
        {
          "audio_selectors": {
            "Audio Selector 1": {
              "default_selection": "DEFAULT"
            }
          },
          "video_selector": {},
          "timecode_source": "ZEROBASED",
          "file_input": "s3://#{source_s3_key}"
        }
      ]
    }
  end
end

I kept the default settings that were set in the AWS console for simplicity, but you can customize them according to your needs. Indeed, it is possible to retrieve the JSON settings from the AWS console for a specific job as shown in the screenshot below.

Periodically check the status of the conversion job and update the database

You must have noticed that we are also calling the CheckHlsVideosJob job after 15 seconds in the ConvertVideoToHlsFormatJob job. This job will periodically check the status of the conversion job and update the database accordingly. It will call itself again and execute 15 seconds later if the conversion is still in progress until it is completed.

# app/jobs/check_hls_videos_job.rb
require "aws-sdk-s3"

class CheckHlsVideosJob < ApplicationJob
  queue_as :default

  rescue_from StandardError do |exception|
    Rails.logger.error(exception)

    @movie.update!(media_convert_status: :internal_error)
  end

  def perform(movie_id:)
    @movie = Movie.find(movie_id)

    inspect_result = MediaConvert::Inspector.new.inspect(job_id: @movie.media_convert_job_id)

    @movie.update!(media_convert_status: inspect_result[:status].downcase.to_sym, media_convert_progress_percentage: inspect_result[:percentage])

    case inspect_result[:status]
    when "SUBMITTED", "PROGRESSING"
      CheckHlsVideosJob.set(wait: 15.seconds).perform_later(movie_id: @movie.id)
    when "COMPLETE"
      attach_hls_video(movie: @movie, output_s3_key: inspect_result[:output_s3_key])
    when "CANCELED", "ERROR"
    end
  end

  private

  def attach_hls_video(movie:, output_s3_key:)
    # Check the `bucket` specified in the `config/storage.yml` (amazon_s3_output_assets)
    # for the HLS video with the key of the attached video
    # and attach the HLS video to the movie
    s3_client = Aws::S3::Client.new(region: "ap-northeast-1", access_key_id: ENV["AWS_ACCESS_KEY_ID"], secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"])

    s3_object = s3_client.head_object(bucket: "tonystrawberry-netflix-output-videos-development", key: output_s3_key)

    ActiveRecord::Base.transaction do
      blob = ActiveStorage::Blob.create!(
        key: output_s3_key,
        filename: output_s3_key.split("/").last,
        content_type: "application/vnd.apple.mpegurl",
        service_name: "amazon_s3_output_assets",
        byte_size: s3_object.content_length,
        checksum: s3_object.etag.gsub(/"/, ""),
      )
      movie.hls_video.attach(blob)
    end
  end
end

This time, we are using the MediaConvert::Inspector class to interact with the AWS Elemental MediaConvert API and get the status of the conversion job. The API will return us the status of the job (SUBMITTED, PROGRESSING, COMPLETE, CANCELED, ERROR), as well as the percentage of the job completion and the output S3 key in case of completion.

# app/services/media_convert/inspector.rb
class MediaConvert::Inspector < MediaConvert::Base
  attr_reader :response, :status, :percentage

  def initialize
    super

    @response = nil
    @status = nil
    @percentage = nil
    @output_s3_key = nil
  end

  def inspect(job_id:)
    @response = client.get_job(id: job_id)

    @status = @response.job.status
    @percentage = @response.job.job_percent_complete
    @output_s3_key = @response.job.settings.output_groups[0].output_group_settings.hls_group_settings.destination

    if @output_s3_key.present?
      @output_s3_key = "#{@output_s3_key.gsub("s3://#{ENV['MEDIA_CONVERT_OUTPUT_BUCKET']}/", "")}.m3u8"
    end

    {
      status: @status,
      percentage: @percentage,
      output_s3_key: @output_s3_key
    }
  end
end

Let's see what files are stored in the tonystrawberry-netflix-output-videos-development bucket after the conversion is completed.

You can see that two kinds of files are stored in the bucket:

  • the playlist files (.m3u8) that reference the video segments. The main file (video.m3u8) is the playlist file that references the other .m3u8 files of different qualities.

  • the video segments (.ts) that are the actual video files in different qualities.

Example of the content of the video.m3u8 file:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-STREAM-INF:BANDWIDTH=1659641,AVERAGE-BANDWIDTH=622281,CODECS="avc1.640015,mp4a.40.2",RESOLUTION=480x270,FRAME-RATE=23.976
video-480x270.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=2193632,AVERAGE-BANDWIDTH=827019,CODECS="avc1.64001e,mp4a.40.2",RESOLUTION=640x360,FRAME-RATE=23.976
video-640x360.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3987679,AVERAGE-BANDWIDTH=1584144,CODECS="avc1.64001f,mp4a.40.2",RESOLUTION=1280x720,FRAME-RATE=23.976
video-1280x720.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=7434262,AVERAGE-BANDWIDTH=3011930,CODECS="avc1.640028,mp4a.40.2",RESOLUTION=1920x1080,FRAME-RATE=23.976
video-1920x1080.m3u8

It contains the references to the video segments in different qualities: 480x270, 640x360, 1280x720 and 1920x1080. Now, let's see the contents of the video-480x270.m3u8 file:

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:11
#EXT-X-MEDIA-SEQUENCE:1
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:10,
video-480x270_00001.ts
#EXTINF:10,
video-480x270_00002.ts
#EXTINF:10,
video-480x270_00003.ts
#EXTINF:10,
video-480x270_00004.ts
...

As expected, the file contains the references to the video segments in the 480x270 quality. The other .m3u8 files contain the references to the exact same video segments in the other qualities. So when the user switches to a different quality, the video player will automatically switch to the corresponding .m3u8 file and download the video segments in that quality without interrupting the video playback.

Watch the video by getting the Cloudfront URL and setting the signed cookies

Finally, we will provide the viewers with the Cloudfront URL to watch the video. Before diving into the code, let me explain why we need authentication and why we are using signed cookies instead of signed URLs to authenticate the users.

In a video application like Netflix, videos are not publicly accessible. You need to be authenticated to watch the videos. In AWS, you can use signed URLs or signed cookies to authenticate the users for accessing the content in Cloudfront.

Signed URLs are URLs that have been signed with a cryptographic signature that allows you to access the content for a limited time. These are good for allowing access to a single resource for a limited time.

But in our case, we want to allow access to multiple files: the playlist file and also all the video segments. For this use case, signed cookies are more appropriate. Signed cookies are cookies that have been signed with a cryptographic signature that allows you to access multiple resources for a limited time.

Signing the cookies

I will explain briefly how to sign the cookies in Ruby on Rails. First, you need to set create a Cloudfront key pair in the AWS console and store both the private key and the generated key ID in your Rails application as environment variables.

CLOUDFRONT_KEY_PAIR_ID=K2X9JOEGGJF68S
CLOUDFRONT_PRIVATE_KEY=LS0tLS1CRUdJTiBQUklWQ... # Base64 encoded private key

Then, you need to set Cloudfront to require signed cookies in the AWS console by setting the Trusted Key Groups to the key pair ID.

Finally, you can sign the cookies in Ruby on Rails using the Aws::CloudFront::CookieSigner class. When signing the cookies, you need to set the policy for the signed cookie. The policy is a JSON object that specifies the resources that the signed cookie can access and the expiration time of the cookie.

Cookies are set in the response headers of the Rails controller by setting the cookies hash.

# app/controllers/movies_controller.rb

# GET /movies/:id
def show
  @movie = Movie.find(params[:id])

  sign_cookie(@movie.hls_video.key)

  @cdn_url = "#{ENV["CLOUDFRONT_URL"]}/#{@movie.hls_video.key}"
end

private

# Set the signed cookie for viewers to access the restricted video content
# This cookie will be used to subsequent requests to the CloudFront distribution
# Reference: https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/CloudFront/CookieSigner.html
def sign_cookie(s3_key)
  signer = Aws::CloudFront::CookieSigner.new(
    key_pair_id: ENV["CLOUDFRONT_KEY_PAIR_ID"],
    private_key: Base64.decode64(ENV["CLOUDFRONT_PRIVATE_KEY"])
  )

  signer.signed_cookie(
    nil,
    policy: policy(s3_key, Time.zone.now + 15.minute)
  ).each do |key, value|
    cookies[key] = {
      value: value,
      domain: :all
    }
  end
end

# The policy for the signed cookie that will be used to access the restricted video content
# Reference: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-setting-signed-cookie-custom-policy.html
def policy(s3_key, expiry)
  # The S3 key looks like this mm57enm1nx2u91ztntbu2idxxror/HLS/mm57enm1nx2u91ztntbu2idxxror.m3u8
  # But I would like the "Resource" to omit the file extension in order to match all files of the same media with different quality prefixes.
  # Example:
  # - mm57enm1nx2u91ztntbu2idxxror/HLS/mm57enm1nx2u91ztntbu2idxxror-270p.m3u8
  # - mm57enm1nx2u91ztntbu2idxxror/HLS/mm57enm1nx2u91ztntbu2idxxror-360p.m3u8
  resource_key = s3_key.split(".")[0]

  {
    "Statement" => [
      {
        "Resource" => "#{ENV["CLOUDFRONT_URL"]}/#{resource_key}*",
        "Condition" => {
          "DateLessThan" => {
            "AWS:EpochTime" => expiry.utc.to_i
          }
        }
      }
    ]
  }.to_json
end

Watch the video

Finally, the viewers can watch the video with a HLS compatible video player by accessing the Cloudfront URL and setting the signed cookies. There are many video players that support HLS such as video.js, HLS.js, Plyr etc. For this example, I will use videojs as it is easy to use and has already a lot of features (playback speed, subtitles, quality switcher etc.).

Javascript code will be added inside a Stimulus controller to initialize the video player and set the video source to the Cloudfront URL.

// app/javascript/controllers/video_player_controller.js
import { Controller } from "@hotwired/stimulus"

// Connects to data-controller="video-player"
export default class extends Controller {
  static targets = [ "video" ]
  static values = { videoUrl: String }

  // Connect the video player when the controller is connected
  connect() {
    const player = videojs(
      this.videoTarget.id,
      {},
      function onPlayerReady() {
        this.play();
      }
    );

    player.hlsQualitySelector();
    player.src({
      src: this.videoUrlValue,
      type: 'application/x-mpegURL',
      withCredentials: true
    });
  }

  // Disconnect the video player when the controller is disconnected
  disconnect() {
    videojs(this.videoTarget.id).dispose();
  }
}

<!-- app/views/movies/show.html.erb -->
<div data-controller="video-player" data-video-player-video-url-value="<%= @cdn_url %>" class="h-lvh bg-black flex justify-center items-center">
  <%= link_to movies_path, class: "absolute left-12 top-12 text-gray-300 cursor-pointer z-10" do %>
    <svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="cursor-pointer w-12 h-12">
      <path stroke-linecap="round" stroke-linejoin="round" d="M10.5 19.5 3 12m0 0 7.5-7.5M3 12h18" />
    </svg>
  <% end %>

  <video
    id="video-player"
    class="video-js absolute w-auto min-w-full min-h-full max-w-none"
    data-video-player-target="video"
    controls
    width="800" height="600"
  >

  </video>
</div>

Voilà! You have now a video streaming application that allows users to upload videos and watch them in different qualities using AWS Elemental MediaConvert and Ruby on Rails. We are using AWS Cloudfront to deliver the video files to the users by caching the content at edge locations. This allows a high number of users to watch the videos with low latency and high availability. Also, we made sure that the video files are secure by using signed cookies to authenticate the users.

Demo

https://www.youtube.com/watch?v=WnZmxHP7vVA
netflixRailsAWSvideo streaming