Large event file uploads | Solvimon

Send usage events to Solvimon in files of millions of lines. You create a file record through the API, upload a gzipped CSV to a short-lived signed URL, and get back an acknowledgement file and a per-line result file.

Why this matters

The event ingestion API handles usage as it happens, and event file uploads carry a CSV inside the request body for files up to 20 MB. Neither fits a nightly export of a million lines or more.

Large file uploads take a different route. The file itself never travels through the API: it goes straight to file storage over a signed URL, so there is no request size limit to work around and no retry of a multi-gigabyte request body. You still need nothing but your normal Solvimon API key, and no Google Cloud account or service account of your own.

Solvimon answers with two files. An ACK arrives within minutes and tells you whether the file was accepted and how many lines it holds. A RESULT arrives when processing finishes and gives one line per input line, so you can resend exactly the lines that failed.

How it works

One file record per upload. The FILE resource is your handle for status, line counts and downloads.
Signed URLs only. Upload and download links are valid for 60 minutes, and you can ask for a new one at any time with your API key.
Nothing starts until you say so. The upload alone processes nothing; the process call does.
Webhooks, not polling. Subscribe to FILE.STATUS_UPDATED and you are told at every step.

Implementation

Via API

Create the file record

Create the file with the create file endpoint, setting source to BUCKET:

Create a bucket file

curl -X POST https://test.api.solvimon.com/v1/files \
  -H "X-API-KEY: <apiKey>" \
  -H "Content-Type: application/json" \
  -d '{
    "reference": "usage-2026-09-01",
    "type": "EVENTS",
    "source": "BUCKET",
    "content_type": "application/gzip",
    "events": { "meter_id": "metr_61Sx9pqJnwo42rzGV7past" }
  }'

Field	Required	Description
`reference`	Yes	Your own unique name for this upload. It shows in Desk and in every webhook.
`type`	Yes	`EVENTS`. The older `METER_DATA` is still accepted and behaves the same, but it is deprecated.
`source`	Yes	`BUCKET`. `DEFAULT` is the in-request upload described in event file uploads.
`content_type`	Yes	`application/gzip`.
`events`	No	A default meter, by `meter_id` or `meter_reference`, for lines that name no meter of their own. With the deprecated `METER_DATA` type the same settings go in `meter_data`; each type takes only its own field.
`processing_type`	No	`DEFAULT` when left out, and the only value a `BUCKET` file takes.

A BUCKET file takes no file_name and no content. Its files are named after the file id and reference instead.

Response 201

{
  "object_type": "FILE",
  "id": "file_8Hq2pqJnwo42rzGV7abcd",
  "reference": "usage-2026-09-01",
  "created_at": "2026-09-01T11:00:00Z",
  "status": "CREATED",
  "type": "EVENTS",
  "processing_type": "DEFAULT",
  "content_type": "application/gzip",
  "source": "BUCKET",
  "upload": {
    "method": "PUT",
    "url": "https://storage.googleapis.com/...?X-Goog-Signature=...",
    "headers": { "Content-Type": "application/gzip" },
    "expire_at": "2026-09-01T12:00:00Z"
  }
}

upload is returned only here, on the create. Send exactly the headers it gives: they are part of the signature.

Upload the file

Upload to the signed URL

curl -X PUT "<upload.url>" \
  -H "Content-Type: application/gzip" \
  --upload-file usage-2026-09-01.csv.gz

The link is checked when the upload starts, so an upload that runs past the 60 minutes still completes. For very large files or unstable connections, use a resumable upload: POST to the same URL with the header x-goog-resumable: start, then upload to the session URL you get back. A session stays valid for 7 days and survives a dropped connection. Google’s client libraries handle this for you.

Uploaded the wrong file? While the file record is CREATED you can upload as often as you like. Each upload replaces the previous one, and only the last upload before you call process is used.

Get a new upload link if yours expired

GET /v1/files/{id}/upload-url returns a fresh link, as often as needed, for as long as the file record is CREATED:

Response

{
  "resource": { "type": "FILE", "id": "file_8Hq2pqJnwo42rzGV7abcd" },
  "method": "PUT",
  "url": "https://storage.googleapis.com/...?X-Goog-Signature=...",
  "headers": { "Content-Type": "application/gzip" },
  "expire_at": "2026-09-01T14:10:00Z"
}

After process the call returns 409. To abandon an upload instead, POST /v1/files/{id}/cancel with no body moves a CREATED file to CANCELLED.

Start processing

When the upload has finished, tell Solvimon with POST /v1/files/{id}/process and no body. The response is the file record, now UPLOADED.

If the file is not in storage yet, the call returns 409. Finish or redo the upload, then call again.
Calling it again before the ACK exists is safe: it answers 200 and re-sends the UPLOADED webhook, and nothing is processed twice. Once the ACK exists it returns 409.
Content already uploaded under another file record, or a file over the size limit, is REJECTED straight away with the reason in the ACK.

Read the ACK

Solvimon checks every line first, with the same validation as the event ingestion API, and against duplicates both inside the file and among events already received. Within minutes the file record moves to VALIDATED or REJECTED, and you get a FILE.STATUS_UPDATED webhook for each change.

Download the ACK

curl https://test.api.solvimon.com/v1/files/{file_id}/ack/download-url \
  -H "X-API-KEY: <apiKey>"

Response

{
  "resource": { "type": "FILE", "id": "file_8Hq2pqJnwo42rzGV7abcd" },
  "expire_at": "2026-09-01T13:00:00Z",
  "url": "https://storage.googleapis.com/...?X-Goog-Signature=..."
}

Follow progress

The file status endpoint shows the line counts growing while the status is PROCESSING:

Response

{
  "id": "file_8Hq2pqJnwo42rzGV7abcd",
  "reference": "usage-2026-09-01",
  "status": "PROCESSING",
  "type": "EVENTS",
  "number_of_lines": 1000000,
  "line_counts": { "processed": 650000, "done": 649988, "not_matched": 0, "duplicate": 0, "error": 12 },
  "bucket_files": {
    "original": { "created_at": "2026-09-01T11:00:01Z", "file_name": "file_8Hq2pqJnwo42rzGV7abcd_usage-2026-09-01.csv.gz" },
    "ack": { "created_at": "2026-09-01T11:01:40Z", "file_name": "file_8Hq2pqJnwo42rzGV7abcd_usage-2026-09-01_ack.csv" }
  }
}

bucket_files lists only the files that exist, so the RESULT is missing here. It is left out entirely until the upload is accepted by process. not_matched and duplicate are only known at the end, so they stay 0 until then.

Read the RESULT and resend what failed

When the file record reaches DONE or DONE_WITH_ERRORS, bucket_files.result is filled at the same moment. Download it with GET /v1/files/{id}/result/download-url.

Put the lines whose status is ERROR, corrected, into a new file with a new reference. Do not resend lines that are DONE, NOT_MATCHED or DUPLICATE.

Via Desk

Creating a bucket file stays API-only, but everything after that is visible in Desk under Files. The list filters on several statuses at once, so you can pull up everything that is still PROCESSING, or everything that ended REJECTED or DONE_WITH_ERRORS.

Opening a file shows:

File details. Reference, name, description, source, meter, upload time, record count and message. A bucket file has no file name, so the page is titled by its reference.
Lines. The counters from line_counts: processed, done, not matched, duplicate and error.
Downloads. Buttons for Original, ACK and Result, each appearing only once that file has been written. A download fetches a fresh signed link and saves under the file’s own name.
Actions. Process on a file that is still CREATED, and Cancel file, which asks for confirmation first. Neither is offered once processing has started, because a run cannot be stopped or undone.
The line table. One row per line with its number, status, reference, event, meter, customer and message, plus the raw CSV line for a line that failed. Meter and customer are links for users who hold METER.VIEW and CUSTOMER.VIEW. The file’s header columns are shown in a tooltip, and the status filter takes several statuses at once, which is what makes a file of a million lines workable: filter to ERROR and DUPLICATE and you see only what needs attention.

Input file format

The input is a gzipped CSV: UTF-8, comma-separated, RFC 4180 quoting, with a header row, one event per line. A byte order mark at the start of the file, as Excel’s “CSV UTF-8” export writes, is ignored. A file with bytes that are not valid UTF-8 is rejected as NOT_UTF8. Once the file has been checked, column_headers on the file record lists the header’s column names.

Input file

reference,customer_reference,meter_reference,timestamp,meter_property.reference.region,meter_value.reference.calls
evt-000001,cust-42,api-calls,2026-09-01T10:00:00Z,eu,12
evt-000002,cust-42,api-calls,2026-09-01T10:05:00Z,us,3

Column	Required	Notes
`reference`	Yes	Unique per event, 10–256 characters. Duplicates are detected on it.
`customer_id` or `customer_reference`	One of the two
`meter_id` or `meter_reference`	One of the two, unless the file record sets a default meter	One file may hold several meters.
`pricing_plan_subscription_id` or `pricing_plan_subscription_reference`	No	Only when a customer has more than one matching subscription.
`timestamp`	No	ISO-8601 with time zone; see timestamp formatting. Defaults to the moment of processing.
`meter_property.reference.<ref>`	Per meter configuration	One column per meter property.
`meter_value.reference.<ref>`	Per meter configuration	Numeric value. For amounts use `.quantity` and `.currency`.

The meter data file endpoint returns a CSV header template for a meter. Add a meter_id or meter_reference column when the file holds more than one meter.

ACK and RESULT files

Both are UTF-8 CSV with a header row and UTC timestamps. The ACK is plain CSV; the RESULT is gzipped. GET /v1/files/{id}/original/download-url returns the file you uploaded.

Each entry in bucket_files carries its file_name, and a download saves under that name: <file id>_<reference>.csv.gz for the original, <file id>_<reference>_ack.csv and <file id>_<reference>_result.csv.gz. The names follow the file record’s current reference, while the ACK’s content keeps the reference it had when the ACK was written.

The ACK

One line, written after the file has been checked and before it is processed.

ACK

file_id,reference,file_size,status,number_of_lines,invalid_lines,duplicate_lines,reason,received_at,acked_at
file_8Hq2pqJnwo42rzGV7abcd,usage-2026-09-01,48213377,VALIDATED,1000000,12,3,,2026-09-01T11:00:02Z,2026-09-01T11:01:40Z

status is VALIDATED, meaning the file will be processed, or REJECTED, meaning nothing is processed.
invalid_lines are lines that fail the event API’s validation, for example an unknown customer or meter, or a missing reference. They are reported as ERROR in the RESULT, with the reason. The rest of the file is processed normally.
duplicate_lines are lines whose reference appears earlier in the file, or was already received. They are reported as DUPLICATE and not processed.
The ACK counts the duplicates known at validation: 0 when there are none, and empty for a rejected file. The final count is in line_counts.duplicate and the RESULT, and it can differ from the ACK when an event with the same reference arrives after the ACK was written.

A rejected file is final: none of its lines is processed and it gets no RESULT. Fix the file and upload it as a new file record with a new reference. The reason column holds the code, and message on the file record repeats it in words, for example MISSING_COLUMN:reference: the header has no reference column.

ACK for a rejected file

file_id,reference,file_size,status,number_of_lines,invalid_lines,duplicate_lines,reason,received_at,acked_at
file_8Hq2pqJnwo42rzGV7abcd,usage-2026-09-01,48213377,REJECTED,,,,MISSING_COLUMN:reference,2026-09-01T11:00:02Z,2026-09-01T11:00:09Z

Reason	Meaning	Found
`NOT_GZIP`	The file is not gzip compressed.	While checking
`NOT_UTF8`	The file is not valid UTF-8. Invalid bytes reject the whole file; they are never replaced.	While checking
`MISSING_COLUMN:<name>`	A required column is not in the header, for example `MISSING_COLUMN:reference`.	While checking
`EMPTY`	The file has a header but no data lines.	While checking
`TOO_LARGE`	Over 2 GB compressed or more than 5,000,000 lines, a single line over 1,048,576 characters, or a header row over 65,536 characters.	At `process` for the size, while checking for the lines
`DUPLICATE_FILE`	The same content was already uploaded under another file record.	At `process`

The RESULT

One line per input line, in the same order.

RESULT

line_item_number,reference,resource_id,status,error_message
1,evt-000001,mdat_a91fpqJnwo42rzGV7a,DONE,
2,evt-000002,mdat_c07epqJnwo42rzGV7b,NOT_MATCHED,No pricing for customer cust-42 at 2026-09-01T10:05:00Z
3,,,ERROR,Missing required column value: reference

line_item_number counts data lines, starting at 1 after the header. resource_id is the Solvimon id of the stored event, and it is empty for ERROR.

Status	Meaning	What to do
`DONE`	Stored and priced.	Nothing.
`NOT_MATCHED`	Stored, but no matching subscription or price at that time. It is priced once the configuration exists.	Check the configuration; do not resend. See unmatched events.
`DUPLICATE`	An event with this reference was already received, and the line was ignored.	Nothing, or resend under a new reference if it is a different event.
`ERROR`	Not stored; see `error_message`.	Fix and resend.

These per-line outcomes are also available as JSON through the file lines endpoint, where a line that is still being processed shows as IMPORTED. Filter with statuses[], which takes several values: ?statuses[]=ERROR&statuses[]=DUPLICATE. The older single-valued status parameter is deprecated. Each line also carries reference, event_id, meter_id and customer_id, and the raw CSV line in data, base64 encoded, for a line that failed.

File statuses and webhooks

Status	Meaning	Final
`CREATED`	Waiting for your upload and for the `process` call.	No
`UPLOADED`	You called `process`; the lines are being checked.	No
`VALIDATED`	Checked and accepted; the ACK is available.	No
`REJECTED`	Not accepted; the ACK is available with the reason.	Yes
`PROCESSING`	Lines are being processed and `line_counts` grows.	No
`DONE`	Every line is `DONE`, `NOT_MATCHED` or `DUPLICATE`; the RESULT is available.	Yes
`DONE_WITH_ERRORS`	At least one line is `ERROR`; the RESULT is available.	Yes
`EXPIRED`	Not processed within 7 days of creation.	Yes
`CANCELLED`	You cancelled the file before calling `process`, and nothing was processed.	Yes
`ERROR`	Processing failed on the Solvimon side. We are alerted and will contact you.	Yes

Subscribe to the FILE.STATUS_UPDATED webhook to be told about every status change. Progress updates send no webhook. A final status and its file always arrive together: when you receive DONE or DONE_WITH_ERRORS, the RESULT can be downloaded straight away.

FILE.STATUS_UPDATED

{
  "created_at": "2026-09-01T11:24:13Z",
  "type": "FILE.STATUS_UPDATED",
  "previous_data": { "id": "file_8Hq2pqJnwo42rzGV7abcd", "status": "PROCESSING" },
  "data": {
    "object_type": "FILE",
    "id": "file_8Hq2pqJnwo42rzGV7abcd",
    "reference": "usage-2026-09-01",
    "status": "DONE_WITH_ERRORS",
    "type": "EVENTS",
    "line_counts": { "done": 999950, "not_matched": 30, "duplicate": 8, "error": 12 },
    "bucket_files": {
      "original": { "created_at": "2026-09-01T11:00:01Z", "file_name": "file_8Hq2pqJnwo42rzGV7abcd_usage-2026-09-01.csv.gz" },
      "ack": { "created_at": "2026-09-01T11:01:40Z", "file_name": "file_8Hq2pqJnwo42rzGV7abcd_usage-2026-09-01_ack.csv" },
      "result": { "created_at": "2026-09-01T11:24:12Z", "file_name": "file_8Hq2pqJnwo42rzGV7abcd_usage-2026-09-01_result.csv.gz" }
    }
  }
}

The webhook carries no download links. Call the download-url endpoint with your API key when you receive it.

Limits and retention

Limit	Value
File size	2 GB compressed
Lines per file	5,000,000
Characters per line	1,048,576, and 65,536 for the header row
Upload and download links	Valid 60 minutes; ask for a new one at any time
Upload window after creating the file record	7 days, then `EXPIRED`
Retention of the original, ACK, RESULT and line results	180 days
The file record, its status and its counts	Kept indefinitely

Edge cases

Forgetting the process call. Nothing is processed. The file record stays CREATED and becomes EXPIRED after 7 days. You can call process at any point before that.
Replacing a file after process. The upload is fixed once process has been called. Create a new file record with the corrected data. If some events were already stored, contact Solvimon about correcting them.
Resending a line that was already DONE. It comes back as DUPLICATE and is not processed again, so resend only the ERROR lines.
Uploading several files at once. Each file needs its own file record. They are processed independently.
Changing the reference. PATCH /v1/files/{id} changes reference, name and description at any status. Every other field is read-only, and a status change is refused and points at process.

Only the last upload before the process call is used, and an upload after that call is ignored. If you are retrying an upload, make sure it has finished before you call process.

See also: Event ingestion API for real-time usage, Event file uploads for files up to 20 MB, Usage file uploads for the Desk walkthrough, and Meters for the columns a meter expects.