FeedReader.Downloader (FeedReader v0.1.0)

Copy Markdown

Streams remote response bodies into owned temporary files while computing the decoded MD5 checksum. Gzip and raw deflate are decoded incrementally; stacked and unknown content encodings are rejected. See download/2 for its options and resource budgets.

Summary

Functions

Removes a downloaded temporary file; safe to call more than once.

Downloads, decodes, checksums, and owns a temporary response file.

Functions

close(map)

@spec close(%{path: Path.t()}) :: :ok

Removes a downloaded temporary file; safe to call more than once.

download(url, opts)

@spec download(String.t(), keyword()) ::
  {:ok, %{path: Path.t(), fetch: FeedReader.Fetch.t()}}
  | {:ok, :not_changed | :not_modified}
  | {:error, term()}

Downloads, decodes, checksums, and owns a temporary response file.

Options

  • :http_client — module implementing FeedReader.Transport; defaults to FeedReader.ReqTransport.
  • :http_options — Req request options. The default transport streams raw response chunks, disables automatic decompression, and limits redirects to its configured limit. A supplied :headers list replaces its default headers.
  • :previous_etag — binary ETag sent verbatim as If-None-Match. A 304 response returns {:ok, :not_modified} without creating a temporary file.
  • :http_retry_attempts — non-negative retry count after the initial request; defaults to the FeedReader.ReqTransport setting. Invalid values use the default.
  • :retry_delay — non-negative delay in milliseconds or an arity-one retry delay function; defaults to the FeedReader.ReqTransport setting. Invalid values use the default.
  • :previous_checksum — checksum of the previous decoded body. An exact match returns {:ok, :not_changed} and removes the temporary file.
  • :max_compressed_bytes — maximum cumulative raw response size; defaults to 52428800 bytes.
  • :max_decoded_bytes — maximum cumulative decompressed body size; defaults to 52428800 bytes.
  • :max_temp_bytes — maximum bytes written to the owned temporary file; defaults to 52428800 bytes.
  • :max_expansion_ratio — maximum decoded-to-compressed byte ratio; defaults to 100.

Invalid resource-limit values use their defaults. Limits are checked as body chunks are processed. Successful downloads return the owned path and a FeedReader.Fetch; callers must pass the download result to close/1 when finished. Errors remove any temporary file created for the request.