Http.Stream
Lets a program send HTTP requests whose bodies are streams, so that it can upload or download something large, or something that arrives slowly, without holding all of it in memory.
This module extends elm/http. A request body can be a
Stream.Readable of bytes, sent as it is read, and a response body can be
handed to the program as a Stream.Readable Bytes, to be read chunk by chunk while it
downloads. Everything that is not about streams stays elm/http's: headers are made with
Http.header, and results are elm/http's Http.Error, Http.Metadata and Http.Response.
This module has its own Body, Expect and Resolver
types, and its own request and task, shaped like elm/http's. The method
of a request is an Http.Method.Method rather than elm/http's String.
A request has two stages, and a failure is reported differently in each. Until the headers of
the final response arrive, the request itself can fail, with Http.BadUrl, Http.Timeout or
Http.NetworkError. Once they have arrived the response is handed over. A failure after that
point, such as the connection closing before the body is complete, reaches the body stream
instead: once the bytes already received have been read, a read of the body fails with
Stream.Failed, which fromStreamFailure reads as Http.NetworkError.
Differences from elm/http:
- A URL with a scheme other than
http:orhttps:fails withHttp.BadUrl. A URL with no scheme fails withHttp.BadUrlon the JavaScript backend. - The
timeoutcovers everything up to the arrival of the final response's headers, including connecting, following redirects and sending the body. Once the headers have arrived the body may take as long as it needs. - There is no
tracker, soHttp.trackandHttp.canceldo not apply. To cancel a request, run it withtaskin a process started withProcess.spawn, and stop it withProcess.killbefore the response headers arrive. After that, cancel the body stream withStream.cancelReadable, which stops a transfer still running. While a read is waiting on the body stream, or a pipe is reading it,cancelReadablefails withLockedand does nothing. - Redirects are followed, at most 20 of them; one more fails with
Http.NetworkError, and a redirect to a URL that is nothttp:orhttps:fails withHttp.BadUrlcarrying the URL of the request. The status, headers andMetadata.urlare those of the final response.
Response header names are lower-cased, and a header that appears more than once has its values
joined with ", " in the order they arrived, as elm/http joins them.
Requests
request :
{ method : Method
, headers : List Header
, url : String
, body : Body
, expect : Expect msg
, timeout : Maybe Float
}
-> Cmd msgBody
type BodyThe body of a request made with this module: nothing, a block of bytes, or a stream.
A body other than emptyBody carries a MIME type, and a non-empty MIME type is
sent as the Content-Type header, unless it holds a CR or LF (or, on the JavaScript backend,
something else Node rejects in a header value). An empty MIME type adds no Content-Type header
on the JavaScript backend; on the native backend libcurl can add
Content-Type: application/x-www-form-urlencoded in its place. A body is made with
emptyBody, stringBody, jsonBody,
bytesBody or streamBody.
emptyBody : BodyA body with nothing in it, and no MIME type, so it adds no Content-Type header of its own.
On the native backend libcurl can still add Content-Type: application/x-www-form-urlencoded
to a POST.
stringBody : String -> String -> BodyMakes a body holding value encoded as UTF-8, sent with the
MIME type mimeType as its Content-Type (an empty
mimeType is treated as Body describes).
A lone surrogate in value is not valid in UTF-8, and the two backends differ: it is sent as
U+FFFD (the bytes EF BF BD) on the JavaScript backend, and in its 3-byte surrogate form on the
native backend.
jsonBody : Value -> BodyMakes a body holding value as JSON text with no whitespace, sent with
Content-Type: application/json.
streamBody : String -> Readable Bytes -> BodyMakes a body that sends what is read from a stream, sent with the
MIME type mimeType as its Content-Type (an empty
mimeType is treated as Body describes). The stream is sent as it is read, so
something large, like a file, can be sent without holding it all in memory:
upload : Stream.Readable Bytes -> Cmd Msg
upload fileStream =
Http.Stream.request
{ method = PUT
, headers = []
, url = "https://example.com/upload"
, body = Http.Stream.streamBody "text/csv" fileStream
, expect = Http.Stream.expectStream Uploaded
, timeout = Nothing
}The end of the stream is the end of the body. The request reads the stream through a pipe, so
the stream is locked, as Stream describes, while the request runs. A stream that is
already locked when the request is sent makes the request fail at once with Http.NetworkError
(on the JavaScript backend, a bad URL is reported first).
If the stream fails, because it is cancelled from its writing side or the resource behind it
fails, a transfer still running is stopped: before the response headers the request fails with
Http.NetworkError, and after them a read of the response body fails with Stream.Failed, which
fromStreamFailure reads as Http.NetworkError.
If instead the transfer ends before the whole stream has been sent (the response is complete,
the request has failed, or it was stopped), the rest is not sent, and the stream is cancelled,
at the latest when it next has a value to send. A 307 or 308 redirect fails with
Http.NetworkError, because a stream cannot be sent twice. Other redirects differ between
backends: on the JavaScript backend a 303, or a 301 or 302 after a POST, is followed as a GET
without a body and any other fails with Http.NetworkError; on the native backend such a request
can wait until its timeout, so set one.
Expect
type Expect msgHow a request turns its response into a message, as elm/http's Http.Expect
does. One is made with expectStream, which keeps only a successful response's
body, or expectStreamResponse, which keeps the body of every response.
expectStream :
Result Error ( Metadata, Readable Bytes ) -> msg
-> Expect msgMakes an Expect that gives toMsg the response's metadata and its body
stream when the status code is in 200–299.
A status code outside 200–299 gives Err (Http.BadStatus code), and the body is not downloaded:
the transfer is stopped as soon as the headers have arrived. A request that fails before the
headers arrive gives Http.BadUrl, Http.Timeout or Http.NetworkError.
Read the body to the end, or cancel it with Stream.cancelReadable once it is no longer needed.
Only about 256 KiB of unread body is held, so a larger body left unread keeps the transfer open.
expectStreamResponse :
Result x a -> msg
-> Response (Readable Bytes) -> Result x a
-> Expect msgMakes an Expect that turns the whole Http.Response into a result with
toResult and gives that to toMsg, as elm/http's Http.expectBytesResponse does.
Unlike expectStream, a response of any status code comes with its body
stream, so the body of, say, a 404 response can be read.
Tasks
task :
{ method : Method
, headers : List Header
, url : String
, body : Body
, resolver : Resolver x a
, timeout : Maybe Float
}
-> Task x aMakes a Task that sends the request, as request does, and succeeds or fails
with what resolver makes of the response. A task can be chained with other tasks, for
example opening a file stream first and then uploading it.
A task is also the way to cancel a request: run it in a process started with Process.spawn,
and stop that process with Process.kill before the response headers arrive, which stops the
transfer. Killing the process after the task has received the response does not stop the
transfer; cancel the body stream with Stream.cancelReadable instead (which fails with
Locked while a read is waiting on the body stream, or a pipe is reading it).
type Resolver x aHow a task turns its response into the task's result: success with an a, or
failure with an x. One is made with streamResolver.
streamResolver :
Response (Readable Bytes) -> Result x a
-> Resolver x aMakes a Resolver that turns the Http.Response into the task's result with
toResult. As with expectStreamResponse, a response of any status
code comes with its body stream.
Body stream failures
fromStreamFailure : Failure -> Maybe ErrorReads a failure of a response body stream as elm/http's Http.Error. Returns Nothing for a
failure that another module's stream made.
case streamError of
Stream.Failed failure ->
case Http.Stream.fromStreamFailure failure of
Just Http.NetworkError ->
"the download was cut off"
_ ->
Stream.failureToString failure
_ ->
...Every failure of a body stream is Http.NetworkError: the connection closing before the body is
complete, the transfer failing, or a streamBody whose stream failed. Once the
response headers have arrived, the request no longer has a timeout, so a body stream never fails
with Http.Timeout. Stream.failureToString gives the details, such as the message the network
library reported.
Sends a request and turns its response into a message, as
expectsays. For example, to download a large file:The
methodis sent asHttp.Method.toStringgives its name, soUNKNOWN "PROPFIND"sendsPROPFIND.The
timeoutis how many milliseconds to wait for the final response's headers before the request fails withHttp.Timeout. It is rounded to whole milliseconds, at least 1, andNothing, or a value of 0 or less, means no timeout. On the JavaScript backend a timeout above 2147483647 milliseconds fails the request after 1 millisecond.A header in
headerswhose name is empty or holds a CR, LF or:, or whose value holds a CR or LF, is not sent. On the JavaScript backend a header that Node rejects, such as one whose name holds a space, is not sent either.