When the server streams data faster than the client can process it, excess data will queue up in the client's memory. This issue is called backpressure, and it can lead to memory overflow errors, or data loss when the client's memory reaches capacity.
In this recipe, you create an API endpoint that:
- Simulates backpressure by generating data faster than a stream can read it
- Handles backpressure by pushing data into a stream as it's needed, rather than as it's ready
Jump to the full example to see the full recipe.
In this case, it will be a generator function that yields a new integer indefinitely
Next, create a method that adds the generator function to a ReadableStream. Using the pull handler, you can prevent new data being added from the generator to the stream if no more data is being requested
Finally, iterate through a loop and read data from the stream. Without the code that checks if the generator is done, the stream would continue taking values from integers() indefinitely, filling up memory. Because the code checks if the generator is done, the stream closes after you iterator as many times as loopCount:
The final file, including the route handler function, will look like this:
If you're not using a framework, you must either add "type": "module" to your package.json or change your JavaScript Functions' file extensions from .js to .mjs