9 minutes

Transload: Easiest Video Upload and Encoding

If you ask me 2 months ago how much hard is handle videos on internet, probably I would say that's not a trouble. Long time ago (>5 years), all websites still used Flashplayer to render videos. Behind scenes, Google and other main companies were implementing video decoders in browsers to avoid this dependency for ever. That promised us a video standard to have a better performance and simplify the development process. The idea wasn't wrong, now videos are rendered taking advantage of graphic device resources and it isn't more complicated than a <video> HTML5 tag. But... we've changed discussion thread from use or not Flash, to what codec should be the standard one. At the moment there isn't a clear answer, if you search on Google "web admitted codecs" you will find 3 options: Mp4, Webm or VP8 and OGV.

Recently in Partelo we switched from Cloudinary to our own storage in Amazon S3 bucket, encoding videos asynchronously thanks to Transloadit and receiving notifications once videos are ready to be played. Then we store results in our S3 bucket and database. Now we save thousands of dollars every month and also reduce scalability costs significantly.

Introduction to Transloadit

This company has more than 10 years of experience providing a solution for a wide variety of Case Studies for startups and large corporations as well.

Transloadit is really Open Source friendly. Besides the amazing asynchronous API to encode videos, images or audio, also they've released many projects to solve some common web problems during uploads like Tus, the Open Protocol for Resumable File Uploads. It's so great that now is the standard protocol of Git LFS. Then they created Uppy a modular Javascript library integrated with Tus to ease file uploads handling for developers. It's richly featured, simple to setup and well-integrated to Transloadit.

Take a look to their 10 years achievements on the blog.

Getting Started

To start working on a real subject, we can create a simple Uppy dashboard which uploads file directly to Transload and then apply an Assembly Template to encode images/audio/videos to your defined instructions. Then in the template you must also configure your exports to store results in AWS S3 Bucket, Azure, Google Drive, FTP server or any available option.

In Transload, any transformation or manipulation is done by a Robot1. They are responsible to create thumbnails, apply filters, manage metadata or import/export files. Take a peek to all available Robots.

Robots are configured in Assembly Templates2 or Assembly Instructions. Templates are basically a JSON array of instructions (aka steps) to apply to the files. Each step or instruction performs some type of encoding, resizing or conversion using a Robot. You would then reference templates via template_id parameter in your API request, keeping your requests lightweight and secure.

ProTip: despite is possible to add Assembly instructions directly in the params field of API Request, you should considerate avoid it and use instead Templates id because embedded secrets, such as S3 and SFTP access credentials, can be seen by anyone in your HTML source!

In the image above, I did a simple flow for a Transload usage case. I recommend you use Uppy to build things faster in a first stage, you can then create your own implementation.

Deeper we see Transload returns a response for every Assembly3. Those responses are named Assembly Notifications, there we find an assembly_id value that we can reuse to reencode again, or make a request to apply subsequent templates to generate subs or manage metadata for example. In our simple case we only need to retrieve the ssl_url (includes https) which is our stored file URL.

Transload recommends build apps or services async if possible because they have to deal with thousands of file transformations sometimes your uploads might be in queues. Nevertheless, they sometimes (if rarely) happen and when that does, your users will have a negative experience if, in your integration, they have to wait on processing.

When using Uppy, that means setting waitForEncoding to false, and making use of Assembly Notifications. With those, your visitors will only need to wait for the file uploading to finish. They do not have wait for the subsequent file conversions to finish as well. On this way, your visitors will bypass any queues, allowing them to move through your app and see results as soon as the encoding has finished.

Build your first Template

Begin creating your Assembly Template in the Transloadit Dashboard. Create an account if you haven't yet, there's a Free plan ideal for testing. As we said, Templates are a JSON list of steps using Robots, so first step is import the file(s) that will be encoded. In this case it's a simple file upload so we need to set first instruction as Robot /upload/handle.

"steps": {
    ":original": {
      "robot": "/upload/handle"
    },
    ...

There are three constraints when using this Robot, you must:

  • not define a use parameter, contrary to all other Robots
  • only use it once in a single set of Assembly Instructions
  • name the Step that it's used in: :original

Next step is for the encoding itself. To create a video flow pipeline, we have to include use: ':original' to point where files are coming to this instruction. The parameter use may be an array too.

In this case, we'll use the Robot /video/encode.

"steps": {
    ":original": ...
    "mp4_encode": {
        "use": [
          ":original"
        ],
        "robot": "/video/encode",
        "result": true,
        "ffmpeg_stack": "v3.3.3",
        "preset": "ipad"
    }
    ...
  • "result": true : this parameter indicates the instruction generate results to be stored/exported.
  • ffmpeg_stack : select the FFmpeg stack version to use for encoding. These versions do not reflect any real FFmpeg versions, instead reflect the Transload own internal (non-semantic) versioning for their custom FFmpeg builds. The current recommendation is to use "v3.3.3"
  • preset : performs conversion using pre-configured settings. This is to prevent you from having to override each of the flash preset's values manually. For a list of video presets, see video presets.

Nice job! Our template is almost done, at the moment is handling file uploads and then performs a simple conversion to Mp4 using the preset ipad, which internally do resizing, change frame rate and audio bitrate to optimize the video file for the web. Now we only need set the place to store all results.

Template credentials

Before implement the export of our Assemblies, take a look to Template Credentials. Template credentials are intended to be reusable and provide a stronger encryption for your cloud storage credentials compared to what Templates already provide for your Assembly instructions.

You can assign them a label to which you can later refer in your Templates in the credentials parameter. Using this label then auto-injects your bucket name, access keys and secrets while your Assembly is running. This saves you from having to duplicate the same access credentials across many Templates.

So go to your Transloadit account and click on "Template Credentials" link under "My account". Choose the proper cloud provider from the dropdown menu at the top of the page and fill out all the fields for bucket names, keys and secrets. Once the Template Credentials are saved, you can use the chosen label in the "credentials" parameter for every import and export Robot.

{
  "steps": {
    "my_import": {
      "robot": "/s3/import",
      "credentials": "my_s3_credentials",
      "path": "/some/path"
    },
    "my_store": {
      "robot": "/youtube/store",
      "use": "my_import",
      "credentials": "my_youtube_credentials"
    }
  }
}


Transloadit Uppy js

Great steps forward! Now to make it work, we only need to implement a Uppy Dashboard using the module for Transload Assemblies. It's very simple once you see de code working:

See the Pen Simple Uppy Dashboard by ayoze (@ayozehd) on CodePen.

In CodePen above, you will see a really starter template to work on Uppy + Transloadit. In Javascript code, I simply added another .use method to include the module Uppy.Transloadit and set the parameter template_id to the template identifier we created before. Finally add your API Key in auth.key, but consider Transloadit recommends enable Signature Authentication to avoid man in the middle or any injection/stealing of your data or billing.

In short the Uppy module performs an API request to Transloadit as you do from backend with their Software Development Kits. In this code, there are some filters to invalidate files bigger than 200MB and only let you upload video files. However it's at client level only, so I would recommend before go to live include those filters in Templates using the Robot /file/filter. Just remember update both sides if you change the conditions.

Test safely that snippet for video uploads, we don't store anything.

Video previews

In Partelo, requirements included a video preview after upload the file because users usually like to see their recorded video just before press "Send". To archive this we created 2 Templates, first one intended for temporal uploads that doesn't "export" anything and return encoded video + original file URL in the Assembly Response. The second template is used once the form is submitted to our servers, we create then a request to perform definitive transformations while set import step as original URL value submitted, and stores Assembly results in our AWS S3 Bucket. Then Transload sends the Assembly Notification to our servers and we save relevant information in database.

Why is this so crazy? Well to render a video, it must be in any browser supported codec first. Usually we only convert to Mp4 because it's supported by almost browsers and devices. Anyway take in consideration convert videos to OGV and Webm to support old devices and browsers too. Briefly the flow is:

  1. We do a first encoding to render the preview as fast as possible avoiding to include extra-processing like watermark.
  2. Temporal Assembly Response returns the original file url + encoded video url (both available only for 24 hours). Original video url is appended to form in a hidden input.
  3. User submits the form and our servers make an API request to Transload using definitive Template with original (temporal) video as base import step.
  4. Transload creates the Assembly (encoding, watermark, metadata...) and send the Assembly Notification to an endpoint which stores results in database.

Wanna take a look to our custom implementation of Uppy & Transloadit in Partelo? You're lucky! Here you have a Gist.

Did you like the reading? Post a comment or rate 5 stars below!

Further reading


  1. Robots import, convert and store your files, and each Assembly Step uses one Robot. You can combine many Robots and put them to work in an Assembly, one Robot handing its results to the next. This allows to create unimagined workflows and encoding pipelines, unique to your use case. 

  2. A Template allows you to save Assembly Instructions securely in your account on transloadit.com. That way, you can hide these instructions from your users. Then you would reference them via a template_id parameter in your API request, making your requests lightweight and secure. 

  3. An Assembly is the execution of file conversion instructions (called Assembly Instructions) on uploaded and/or imported files. Each Assembly has a unique assembly_id identifier that can be used to refer to the Assembly in subsequent API requests. 

1.0