# `ReqS3`

ReqS3 handles a custom `s3://` url scheme. Example requests are:

```elixir
# list buckets
Req.get!(req, url: "s3://")

# list objects
Req.get!(req, url: "s3://#{bucket}")

# get object
Req.get!(req, url: "s3://#{bucket}/#{key}")

# put object
Req.put!(req, url: "s3://#{bucket}/#{key}")
```

The responses for listing buckets and objects are automatically decoded.

## Usage

```elixir
Mix.install([
  {:req, "~> 0.5.0"},
  {:req_s3, "~> 0.2.3"}
])

req = Req.new() |> ReqS3.attach()

Req.get!(req, url: "s3://ossci-datasets").body
#=>
# %{
#   "ListBucketResult" => %{
#     "Contents" => [
#       %{"Key" => "mnist/", ...},
#       %{"Key" => "mnist/t10k-images-idx3-ubyte.gz", ...},
#       ...
#     ],
#     "Name" => "ossci-datasets",
#     ...
#   }
# }

body = Req.get!(req, url: "s3://ossci-datasets/mnist/t10k-images-idx3-ubyte.gz").body
<<_::32, n_images::32, n_rows::32, n_cols::32, _body::binary>> = body
{n_images, n_rows, n_cols}
#=> {10_000, 28, 28}
```

## Examples

  * [MNIST](examples/mnist.livemd)

  * [S3 Direct Upload](examples/upload.livemd)

  * [MINIO](examples/minio.exs)

```sh
$ docker run -p 9000:9000 \
   -e MINIO_ROOT_USER=minio \
   -e MINIO_ROOT_PASSWORD=minio123 \
   minio/minio server /data
```

### Pre-signing

ReqS3 can be used to presign URLs:

```elixir
options = [
  access_key_id: System.fetch_env!("AWS_ACCESS_KEY_ID"),
  secret_access_key: System.fetch_env!("AWS_SECRET_ACCESS_KEY")
]

req = Req.new() |> ReqS3.attach(aws_sigv4: options)
%{status: 200} = Req.put!(req, url: "s3://bucket1/key1", body: "Hello, World!")

presigned_url = ReqS3.presign_url([bucket: "bucket1", key: "key1"] ++ options)
#=> "https://bucket1.s3.amazonaws.com/key1?X-Amz-Algorithm=AWS4-HMAC-SHA256&..."

Req.get!(presigned_url).body
#=> "Hello, World!"
```

and form uploads:

```elixir
form = ReqS3.presign_form([bucket: "bucket1", key: "key1"] ++ options)
%{status: 204} = Req.post!(form.url, form_multipart: [file: "Hello, World!"] ++ form.fields)

Req.get!(presigned_url).body
#=> "Hello, World!"
```

## Environment Variables

ReqS3 supports the following standardised system environment variables:

  * `AWS_ACCESS_KEY_ID`

  * `AWS_SECRET_ACCESS_KEY`

  * `AWS_REGION`

  * `AWS_ENDPOINT_URL_S3`

# `attach`

Attaches the plugin.

# `handle_s3_url`

Handles the `s3://` URL scheme.

This request step is automatically added on `ReqS3.attach(request)`.

See module documentation for usage examples.

## Request Options

  * `:aws_endpoint_url_s3` - if set, the endpoint URL for S3-compatible services.
    If `AWS_ENDPOINT_URL_S3` system environment variable is set, it is considered first.

# `presign_form`

Returns presigned form for upload.

## Options

  * `:access_key_id` - the AWS access key id. Defaults to the value of `AWS_ACCESS_KEY_ID`
    system environment variable.

  * `:secret_access_key` - the AWS secret access key. Defaults to the value of
    `AWS_SECRET_ACCESS_KEY` system environment variable.

  * `:region` - if set, AWS region. Defaults to the value of `AWS_REGION` system environment
    variable, then `"us-east-1"`.

  * `:bucket` - the S3 bucket.

  * `:key` - the S3 bucket key.

  * `:endpoint_url` - if set, the endpoint URL for S3-compatible services. If
    `AWS_ENDPOINT_URL_S3` system environment variable is set, it is considered first.

  * `:content_type` - if set, the content-type of the uploaded object.

  * `:max_size` - if set, the maximum size of the uploaded object.

  * `:expires_in` - the time in milliseconds before the signed upload expires. Defaults to 1h
    (`60 * 60 * 1000` milliseconds).

  * `:datetime` - the request datetime, defaults to `DateTime.utc_now(:second)`.

## Examples

    iex> options = [
    ...>   access_key_id: System.fetch_env!("AWS_ACCESS_KEY_ID"),
    ...>   secret_access_key: System.fetch_env!("AWS_SECRET_ACCESS_KEY"),
    ...>   bucket: "bucket1",
    ...>   key: "key1",
    ...>   expires_in: :timer.hours(1)
    ...> ]
    iex> form = ReqS3.presign_form(options)
    iex> form.url
    "https://bucket1.s3.amazonaws.com"
    iex> form.fields
    [
      {"key", "key1"},
      {"policy", "eyJjb25kaXRpb25z...ifQ=="},
      {"x-amz-algorithm", "AWS4-HMAC-SHA256"},
      {"x-amz-credential", "AKIA.../20240528/us-east-1/s3/aws4_request"},
      {"x-amz-date", "20240528T105226Z"},
      {"x-amz-server-side-encryption", "AES256"},
      {"x-amz-signature", "465315d202fbb2ce081f79fca755a958a18ff68d253e6d2a611ca4b2292d8925"}
    ]
    iex> Req.post!(form.url, form_multipart: form.fields ++ [{"file", "test"}])
    iex> Req.get!("https://bucket1.s3.amazonaws.com/key1").body
    "test"

See `Req.Steps.encode_body/1` for more information on multipart uploads.

# `presign_url`

Returns a presigned URL for fetching bucket object contents.

## Options

  * `:access_key_id` - the AWS access key id. Defaults to the value of `AWS_ACCESS_KEY_ID`
    system environment variable.

  * `:secret_access_key` - the AWS secret access key. Defaults to the value of
    `AWS_SECRET_ACCESS_KEY` system environment variable.

  * `:region` - the AWS region. Defaults to the value of `AWS_REGION` system environment
    variable, then `"us-east-1"`.

  * `:method` - the HTTP method, defaults to `:get`.

  * `:url` - the URL to presign, for example: `"https://{bucket}.s3.amazonaws.com/{key}"`,
    `s3://{bucket}/{key}`, etc. `s3://` URL uses `:endpoint_url` option described below.

    Instead of passing the `:url` option, you can instead pass `:bucket` and `:key` options
    which will generate `https://{bucket}.s3.amazonaws.com/{key}` (or
    `{endpoint_url}/{bucket}/{key}`).

  * `:endpoint_url` - if set, the endpoint URL for S3-compatible services. If
    `AWS_ENDPOINT_URL_S3` system environment variable is set, it is considered first.

  * `:expires` - the number of seconds after which the URL should expire. Defaults to
    `86400` (1 day).

## Examples

Note: This example assumes `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables
are set.

    iex> req = Req.new() |> ReqS3.attach()
    iex> bucket = System.fetch_env!("BUCKET_NAME")
    iex> key = "key1"
    iex> %{status: 200} = Req.put!(req, url: "s3://#{bucket}/#{key}", body: "Hello, World!")
    iex> url = ReqS3.presign_url(bucket: bucket, key: key)
    iex> url =~ "https://#{bucket}.s3.amazonaws.com/#{key}?X-Amz-Algorithm=AWS4-HMAC-SHA256&"
    true
    iex> %{status: 200, body: body} = Req.get!(url)
    iex> body
    "Hello, World!"

---

*Consult [api-reference.md](api-reference.md) for complete listing*