Services should have a way to stream data in chunks
Description
Release Notes
Activity
Terence Yim January 9, 2016 at 7:53 AMEdited
PR for addition of HttpContentProducer
Andreas Neumann November 11, 2015 at 8:31 AM
This looks pretty good. I agree that the fluent style is nice but out of sync.
One thing I am concerned about is whether it is intuitive for a developer that the method is called onSend(). This appears like a callback that is driven by a sending of a chunk of data from outside. But the truth is that this method is providing the chunk. If the method were named nextChunk(), or provideChunk(), it would be clear that this method is the one actually produces the chunk to be sent.
Terence Yim November 11, 2015 at 1:00 AM
We can use the fluent style, however it would be a bit out-of-sync of the current API style.
Alvin Wang November 11, 2015 at 12:53 AM
I prefer having both the `send(HttpContentProducer, ..)` and `send(Location, ..)` variants. This follows the pattern of the existing service API, but how about a fluent style to reduce code duplication?
responder.prepare()
.location(someLocation)
.headers(ImmutableMap.of("foo", "bar"))
.send();
responder.prepare()
.error(400, "Unable to read path")
.send();Terence Yim November 10, 2015 at 10:39 PM
A slightly alternate API on the HttpServiceResponder is to only have the send(HttpContentProducer variants. The support of sending Location will be done through a LocationHttpContentProducer implementation that is provided through cdap-api. The advantage of this is it's easier to expand the API by providing more implementations of HttpContentProducer, such as FileHttpContentProducer, URLHttpContentProducer... etc.
Details
Details
Assignee
Reporter
Affects versions
Components
Fix versions
Priority
Major
Suppose you want to upload a large file through a service endpoint. Right now, the entire request gets read into memory before the handler is called. It would be better to support chunking. The same would be good for the response of a handler.