Step-by-Step Guide: Upload a Binary File to S3 Using AWS SDK for Node.js (Updated Documentation Reference)

Amazon S3 (Simple Storage Service) is a scalable object storage service offered by AWS, widely used for storing and retrieving files of all types—including binary files like images, PDFs, videos, and executables. Uploading binary files to S3 programmatically is a common task in backend development, and the AWS SDK for Node.js simplifies this process.

In this guide, we’ll focus on uploading binary files to S3 using the latest AWS SDK for Node.js (v3), which is modular, lightweight, and optimized for modern JavaScript. We’ll cover everything from setup to handling edge cases like large files, with direct references to AWS’s updated documentation. By the end, you’ll have a working implementation to upload binary files (e.g., images, PDFs) to S3 reliably.

Table of Contents#

  1. Prerequisites
  2. Setting Up AWS Credentials
  3. Project Setup
  4. Installing Dependencies
  5. Understanding AWS SDK v3 for S3
  6. Step-by-Step File Upload Implementation
  7. Handling Edge Cases & Errors
  8. Testing the Upload
  9. References

Prerequisites#

Before you begin, ensure you have the following:

  • Node.js & npm: Installed (v14+ recommended). Download here.
  • AWS Account: With an S3 bucket created. Create a bucket (note the bucket name and region).
  • AWS IAM Permissions: A user/role with s3:PutObject permission for the target bucket. Learn more about IAM policies for S3.
  • Basic JavaScript/Node.js Knowledge: Familiarity with async/await and module imports.

Setting Up AWS Credentials#

The AWS SDK requires credentials to authenticate with AWS. For local development, the easiest way is to use environment variables or the AWS credentials file.

Option 1: Environment Variables#

Set these variables in your terminal or use a .env file (we’ll use dotenv later to load them):

AWS_ACCESS_KEY_ID=your_access_key  
AWS_SECRET_ACCESS_KEY=your_secret_key  
AWS_REGION=us-east-1  # e.g., us-east-1, eu-west-1  
S3_BUCKET_NAME=your-bucket-name

Option 2: AWS Credentials File#

Create/modify ~/.aws/credentials (Linux/macOS) or C:\Users\<User>\.aws\credentials (Windows):

[default]  
aws_access_key_id = your_access_key  
aws_secret_access_key = your_secret_key

And ~/.aws/config:

[default]  
region = us-east-1

Best Practice: For production, use IAM roles (e.g., on EC2, ECS, or Lambda) instead of access keys. IAM Roles for EC2.

Project Setup#

Let’s create a new Node.js project:

  1. Create a project folder and navigate to it:

    mkdir s3-binary-upload && cd s3-binary-upload

  2. Initialize npm (accept defaults):

    npm init -y

  3. Create an entry file (e.g., upload.js):

    touch upload.js  # Linux/macOS  
# or  
type nul > upload.js  # Windows

Installing Dependencies#

AWS SDK v3 is modular, so we only install the packages we need. For S3 uploads, we’ll use:

  • @aws-sdk/client-s3: Core S3 client for API operations.
  • @aws-sdk/lib-storage: Optional, for simplified multipart uploads (large files).
  • dotenv: To load environment variables (if using .env).

Install them via npm:

npm install @aws-sdk/client-s3 @aws-sdk/lib-storage dotenv

Understanding AWS SDK v3 for S3#

Unlike SDK v2 (monolithic), SDK v3 uses modular packages. For S3, key components include:

  • S3Client: Configures and manages S3 API requests (region, credentials, etc.).
  • PutObjectCommand: The API command to upload an object to S3 (for files ≤ 5GB).
  • Upload (from @aws-sdk/lib-storage): A higher-level utility for multipart uploads (files > 5GB).

We’ll use S3Client + PutObjectCommand for basic uploads and Upload for large files.

Step-by-Step File Upload Implementation#

Basic Upload (Small Files ≤ 5GB)#

For files smaller than 5GB, PutObjectCommand is sufficient. Here’s how to implement it:

Step 1: Load Environment Variables#

Create a .env file in your project root and add your AWS/S3 details:

AWS_ACCESS_KEY_ID=your_access_key  
AWS_SECRET_ACCESS_KEY=your_secret_key  
AWS_REGION=us-east-1  
S3_BUCKET_NAME=your-bucket-name

Step 2: Import Dependencies#

In upload.js, import the required modules:

require('dotenv').config(); // Load .env variables  
const { S3Client, PutObjectCommand } = require('@aws-sdk/client-s3');  
const fs = require('fs/promises'); // For reading files asynchronously  
const path = require('path'); // For handling file paths

Step 3: Configure the S3 Client#

Initialize S3Client with your AWS region and credentials (loaded from environment variables):

const s3Client = new S3Client({  
  region: process.env.AWS_REGION,  
  credentials: {  
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,  
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,  
  },  
});

Step 4: Define the Upload Function#

Create an async function to read a binary file from disk and upload it to S3:

/**  
 * Uploads a binary file to S3.  
 * @param {string} filePath - Path to the local file (e.g., './image.jpg').  
 * @param {string} s3Key - Desired key (path/name) for the file in S3 (e.g., 'uploads/image.jpg').  
 * @param {string} contentType - MIME type of the file (e.g., 'image/jpeg').  
 */  
async function uploadBinaryFileToS3(filePath, s3Key, contentType) {  
  try {  
    // Read the binary file from disk (returns a Buffer)  
    const fileContent = await fs.readFile(filePath);  
 
    // Define upload parameters  
    const params = {  
      Bucket: process.env.S3_BUCKET_NAME,  // Your bucket name  
      Key: s3Key,                          // Name/key in S3  
      Body: fileContent,                   // File buffer (binary data)  
      ContentType: contentType,            // MIME type (e.g., 'image/jpeg', 'application/pdf')  
      // Optional: Set ACL (e.g., 'public-read' for public access)  
      // ACL: 'public-read'  
    };  
 
    // Create and send the PutObjectCommand  
    const command = new PutObjectCommand(params);  
    const response = await s3Client.send(command);  
 
    console.log('File uploaded successfully!', response);  
    return response;  
 
  } catch (error) {  
    console.error('Error uploading file:', error);  
    throw error; // Re-throw to handle upstream  
  }  
}

Step 5: Execute the Upload#

Call the function with a sample binary file (e.g., ./test-image.jpg):

// Example usage  
(async () => {  
  const localFilePath = path.join(__dirname, 'test-image.jpg'); // Path to your file  
  const s3Key = 'uploads/test-image.jpg'; // Key in S3 (e.g., 'folder/filename.jpg')  
  const contentType = 'image/jpeg'; // MIME type  
 
  await uploadBinaryFileToS3(localFilePath, s3Key, contentType);  
})();

Multipart Upload (Large Files > 5GB)#

For files larger than 5GB, use Upload from @aws-sdk/lib-storage (simplifies multipart uploads, which split files into chunks):

Step 1: Import Upload#

Add to the top of upload.js:

const { Upload } = require('@aws-sdk/lib-storage');

Step 2: Define a Multipart Upload Function#

Replace or extend the earlier function with:

/**  
 * Upload large files (>5GB) to S3 using multipart upload.  
 * @param {string} filePath - Path to the local file.  
 * @param {string} s3Key - Key in S3.  
 * @param {string} contentType - MIME type.  
 */  
async function uploadLargeFileToS3(filePath, s3Key, contentType) {  
  try {  
    // Create a read stream for the file (more memory-efficient than loading the entire file)  
    const fileStream = fs.createReadStream(filePath);  
 
    // Configure multipart upload  
    const upload = new Upload({  
      client: s3Client,  // Reuse the S3Client instance  
      params: {  
        Bucket: process.env.S3_BUCKET_NAME,  
        Key: s3Key,  
        Body: fileStream,  
        ContentType: contentType,  
        // Optional: ACL, Metadata, etc.  
      },  
      // Optional: Customize chunk size (default: 5MB)  
      // partSize: 10 * 1024 * 1024, // 10MB chunks  
    });  
 
    // Track upload progress (optional)  
    upload.on('httpUploadProgress', (progress) => {  
      console.log(`Upload progress: ${Math.round((progress.loaded / progress.total) * 100)}%`);  
    });  
 
    // Execute the upload  
    const response = await upload.done();  
    console.log('Large file uploaded successfully!', response);  
    return response;  
 
  } catch (error) {  
    console.error('Error uploading large file:', error);  
    throw error;  
  }  
}

Step 3: Use the Multipart Function#

Update the example usage to test with a large file:

(async () => {  
  // For large files (uncomment and adjust paths)  
  const largeFilePath = path.join(__dirname, 'large-video.mp4');  
  const largeS3Key = 'videos/large-video.mp4';  
  const largeContentType = 'video/mp4';  
 
  await uploadLargeFileToS3(largeFilePath, largeS3Key, largeContentType);  
})();

Handling Edge Cases & Errors#

Common Errors to Handle#

  • AccessDenied: Check IAM permissions (ensure s3:PutObject is allowed).
  • NoSuchBucket: Verify the bucket name and region.
  • EntityTooLarge: Use multipart uploads for files > 5GB.
  • File Not Found: Ensure the local file path is correct.

Best Practices#

  • Validate File Size: Check file size before upload and choose PutObject or Upload dynamically.
  • Set ContentType: Critical for browsers to render files correctly (e.g., image/png vs application/octet-stream). Use mime-types package to auto-detect MIME types: 
    npm install mime-types
    Then: 
    const mime = require('mime-types');  
const contentType = mime.lookup(filePath) || 'application/octet-stream';
  • Clean Up Streams: Ensure file streams are closed on error (SDK v3 handles this by default, but good to confirm).

Testing the Upload#

  1. Prepare a Test File: Place a binary file (e.g., test-image.jpg) in your project folder.
  2. Update .env: Ensure all variables (bucket name, region, credentials) are correct.
  3. Run the Script: 
    node upload.js
  4. Verify in S3 Console: Navigate to your bucket in the AWS S3 Console and check for the uploaded file (e.g., uploads/test-image.jpg).

References#

By following this guide, you’ve learned to upload binary files to S3 using AWS SDK v3 for Node.js, with support for both small and large files. The modular design of SDK v3 ensures your app stays lightweight, and the examples here align with AWS’s latest best practices. Let us know in the comments if you encountered issues or have questions!

2026-07