Skip to content

node-fs.mdx

Latest commit

 

History

History
484 lines (373 loc) · 14.4 KB

node-fs.mdx

File metadata and controls

484 lines (373 loc) · 14.4 KB
Raw
title Node:FS
sidebarTitle node:fs
description Read and write files in Edge Scripts using the Node.js-compatible file system API
No breaking changes are expected, but additional features will be implemented soon.

Overview

Edge Scripting provides support for the Node.js file system API through the node:fs and node:fs/promises modules. This allows you to read, write, and manipulate files and directories within your Edge Scripts.

The file system API is based on Deno's sandboxed environment with Node.js compatibility, supporting both the modern Promise-based API (node:fs/promises) and the traditional callback-based API (node:fs).

We recommend using the Promise-based API for cleaner async/await syntax.

Virtual File System

Each script has now access to a Virtual File System that allow to handle file based operations.

This file system lives on the Virtual Memory available of the worker.

Currently the file directory is composed of two folder:

/ # Root directory
├── home/
│   └── user/ # Your current working directory, R/W access
└── tmp/ # Temporary directory, R/W access

You can create directories anywhere. Permissions are enforced for the owner (user) but not for group or others. All files are opened in w+.

We suggest you to avoid creating directory in root directory `/` as we are going to create new folders with differents functions, like `/dev/*` and other read only directories.

In the Virtual File System, paths are limited to 4096 chars, for instance foo/bar is 7 characters. It's also limited to 48 segments (a/b/c is 3 segments).

Capacity checks for creating, copying and moving are done before the operation. This can result in trying to copy a large file resulting in an out of memory error, even though there is space left to create new files or directories.

Importing the Module

EdgeScripting supports both the modern Promise-based API and the traditional callback-based API.

Promise-based API (Recommended)

import * as fs from "node:fs/promises";

// Use with async/await
const data = await fs.readFile("/tmp/file.txt", "utf-8");

Callback-based API

import * as fs from "node:fs";

// Use with callbacks
fs.readFile("/tmp/file.txt", "utf-8", (err, data) => {
  if (err) {
    console.error("Error:", err);
    return;
  }
  console.log("Data:", data);
});

Limitations

EdgeScripting's file system implementation operates in a sandboxed environment with limitations. Understanding these constraints is essential for building reliable applications.

Always handle file system errors gracefully. The sandboxed environment may behave differently from traditional Node.js environments.

Symbolic Links and Hard Links

Symbolic links are supported. symlink(), symlinkSync(), readlink(), readlinkSync(), and lstat() work as expected.

Hard links are not supported. `link()` and `linkSync()` will throw errors.

File Statistics - Timestamps

All Stats timestamp properties are supported and return correct values:

Reliable Stats properties:

  • size - File size in bytes
  • isFile() - Check if entry is a file
  • isDirectory() - Check if entry is a directory
  • atime - Last access time
  • mtime - Last modification time
  • ctime - Last status change time
  • atimeMs, mtimeMs, ctimeMs - Millisecond timestamps
  • birthtime - File creation time

File and Directory Permissions

File and directory permissions are supported for the owner (user) bits. chmod() and chmodSync() work, and Stats.mode reflects the permissions you set. access() respects owner-level read, write, and execute bits.

**Group and others** permission bits are accepted but have no effect: the sandboxed environment always runs as the file owner (uid=1000, gid=1000), so only the user/owner bits matter.

chown() and chownSync() are implemented for compatibility, but ownership is not enforced. Changing the uid or gid has no effect on file access or permissions.

File Watching

File system watching is not supported.

The following operations are unavailable: - `watch()` - Watch for file changes - `watchFile()` - Poll for file changes - `unwatchFile()` - Stop watching - `FSWatcher` class

Workaround: Use polling with stat() if you need to detect changes, but be mindful of performance implications and CPU time limits.

Performance Considerations

**Memory limits:** The whole Virtual FileSystem lives inside your script memory. The current file system limits of your script is set up to 64MB. Reading large files may cause memory exhaustion if you store it in memory. Leverage streaming to avoid allocating too much memory at a time.

CPU time: File I/O counts toward the 30-second CPU time limit per request.

Best practices:

  • Stream large files instead of reading entirely into memory
  • Clean up temporary files to avoid storage bloat
  • See Limits for more details on resource constraints

Quickstart

Here are practical examples showing common file system patterns in EdgeScripting.

Example 1: Reading a File

import * as fs from "node:fs/promises";
import * as BunnySDK from "@bunny.net/edgescript-sdk";

BunnySDK.net.http.serve(async (request: Request) => {
  try {
    // Read file as UTF-8 string
    const content = await fs.readFile("/tmp/data.txt", "utf-8");

    return new Response(content, {
      headers: { "content-type": "text/plain" }
    });
  } catch (error) {
    return new Response(`Error reading file: ${error.message}`, {
      status: 500
    });
  }
});

Example 2: Writing a File

import * as fs from "node:fs/promises";
import * as BunnySDK from "@bunny.net/edgescript-sdk";

BunnySDK.net.http.serve(async (request: Request) => {
  const data = await request.json();

  try {
    // Write JSON data to file
    await fs.writeFile(
      "/tmp/output.json",
      JSON.stringify(data, null, 2),
      "utf-8"
    );

    return new Response("File written successfully", { status: 201 });
  } catch (error) {
    return new Response(`Error writing file: ${error.message}`, {
      status: 500
    });
  }
});

Example 3: Working with Directories

import * as fs from "node:fs/promises";
import * as BunnySDK from "@bunny.net/edgescript-sdk";

BunnySDK.net.http.serve(async (request: Request) => {
  try {
    // Create directory (recursive option creates parent directories)
    await fs.mkdir("/tmp/myapp/data", { recursive: true });

    // Write a file in the new directory
    await fs.writeFile("/tmp/myapp/data/config.json", "{}", "utf-8");

    // List directory contents
    const files = await fs.readdir("/tmp/myapp/data");

    return new Response(JSON.stringify({ files }), {
      headers: { "content-type": "application/json" }
    });
  } catch (error) {
    return new Response(`Error: ${error.message}`, { status: 500 });
  }
});

Example 4: Error Handling Patterns

import * as fs from "node:fs/promises";
import * as BunnySDK from "@bunny.net/edgescript-sdk";

BunnySDK.net.http.serve(async (request: Request) => {
  const filepath = "/tmp/cache/data.txt";

  try {
    // Check if file exists by trying to access it
    await fs.access(filepath);

    // File exists, read it
    const content = await fs.readFile(filepath, "utf-8");
    return new Response(content, {
      headers: { "x-cache": "hit" }
    });
  } catch (error) {
    if (error.code === "ENOENT") {
      // File doesn't exist, create it
      const defaultContent = "Default cached data";
      await fs.writeFile(filepath, defaultContent, "utf-8");

      return new Response(defaultContent, {
        status: 201,
        headers: { "x-cache": "miss" }
      });
    }

    // Re-throw other errors
    throw error;
  }
});

Example 5: Using FileHandle for Chunked Reading

This is not an optimized script at all. This is more to show an example of chunked reading, please avoid using something similar in production. 
import * as fs from "node:fs/promises";
import * as BunnySDK from "@bunny.net/edgescript-sdk";

// Global promise to track the download operation
let downloadPromise: Promise<void> | null = null;
const filePath = "/tmp/large-file.txt";

async function downloadLoremIpsum() {
  try {
    const response = await fetch('https://lorem-api.com/api/lorem', {
      method: 'GET',
      headers: {
        'Content-Type': 'application/json'
      }
    });

    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }

    const content = await response.text();
    
    // Calculate how many times we need to repeat to reach ~10MB
    const targetSize = 10 * 1024 * 1024; // 10MB in bytes
    const contentSize = Buffer.byteLength(content, 'utf-8');
    const repeatCount = Math.ceil(targetSize / contentSize);
    
    // Create the repeated content
    let finalContent = '';
    for (let i = 0; i < repeatCount; i++) {
      finalContent += content;
    }
    
    // Write to file
    await fs.writeFile(filePath, finalContent, 'utf-8');
  } catch (error) {
    // Clear the promise on error so it can be retried
    downloadPromise = null;
    throw error;
  }
}

async function ensureFileExists() {
  try {
    // Check if file exists
    await fs.access(filePath);
  } catch (error) {
    // File doesn't exist, initiate download
    if (!downloadPromise) {
      // Start download and store the promise
      downloadPromise = downloadLoremIpsum();
    }
    // Wait for the download to complete (whether we started it or another request did)
    await downloadPromise;
  }
}

BunnySDK.net.http.serve(async (request: Request) => {
  try {
    // Ensure file exists (download if necessary, blocking until complete)
    await ensureFileExists();

    // Create SSE stream
    const stream = new ReadableStream({
      async start(controller) {
        const encoder = new TextEncoder();
        let fileHandle;
        
        try {
          // Open file for reading
          fileHandle = await fs.open(filePath, "r");
          
          // Get file stats to know the total size
          const stats = await fileHandle.stat();
          const totalSize = stats.size;
          
          // Send initial event with file info
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({
            type: "start",
            totalSize,
            chunkSize: 1024,
            timestamp: new Date().toISOString()
          })}\n\n`));
          
          const buffer = new Uint8Array(1024);
          let position = 0;
          let chunkNumber = 0;
          
          // Read file 1KB at a time
          while (position < totalSize) {
            const { bytesRead } = await fileHandle.read(buffer, 0, 1024, position);
            
            if (bytesRead === 0) break; // End of file
            
            // Convert to string
            const content = new TextDecoder().decode(buffer.slice(0, bytesRead));
            
            // Send chunk as SSE event
            controller.enqueue(encoder.encode(`data: ${JSON.stringify({
              type: "chunk",
              chunkNumber,
              bytesRead,
              position,
              content,
              progress: ((position + bytesRead) / totalSize * 100).toFixed(2) + "%",
              timestamp: new Date().toISOString()
            })}\n\n`));
            
            position += bytesRead;
            chunkNumber++;
            
            // Small delay to prevent overwhelming the client
            await new Promise(resolve => setTimeout(resolve, 10));
          }
          
          // Send completion event
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({
            type: "complete",
            totalChunks: chunkNumber,
            totalBytesRead: position,
            timestamp: new Date().toISOString()
          })}\n\n`));
          
        } catch (error) {
          // Send error event
          controller.enqueue(encoder.encode(`data: ${JSON.stringify({
            type: "error",
            message: error.message,
            timestamp: new Date().toISOString()
          })}\n\n`));
        } finally {
          // Close file handle
          await fileHandle?.close();
          controller.close();
        }
      }
    });
    
    return new Response(stream, {
      headers: {
        "Content-Type": "text/event-stream",
        "Cache-Control": "no-cache",
        "Connection": "keep-alive",
        "X-Accel-Buffering": "no"
      }
    });
    
  } catch (error) {
    return new Response(`Error: ${error.message}`, { status: 500 });
  }
});

Example 6: Symbolic Links

import * as fs from "node:fs/promises";
import * as BunnySDK from "@bunny.net/edgescript-sdk";

BunnySDK.net.http.serve(async (request: Request) => {
  const original = "/tmp/original.txt";
  const link = "/tmp/link.txt";

  try {
    await fs.writeFile(original, "hello from original", "utf-8");

    // Create a symbolic link pointing to the original file
    await fs.symlink(original, link);

    // Reading through the symlink is transparent
    const content = await fs.readFile(link, "utf-8");

    // lstat inspects the link itself, not the target
    const linkStats = await fs.lstat(link);
    const target = await fs.readlink(link);

    return new Response(
      JSON.stringify({ content, isSymlink: linkStats.isSymbolicLink(), target }),
      { headers: { "content-type": "application/json" } }
    );
  } catch (error) {
    return new Response(`Error: ${error.message}`, { status: 500 });
  }
});

References