Vinova Lab · Engineering

Storage connectors

What every connector challenge requires, whichever provider it names.

Version 1.030 August 2026SHA-256 4f13c6fa7bb4b88ee978423d8163891c7ff8b87d037d42946be79d5b262cd7e5

Storage connectors

Every storage connector challenge is reviewed against this document. The brief of a single challenge says which provider and what is specific to it; this page says everything that is the same for all of them.

Every rule has a number. A review that returns work cites the rule it fails — §4.2, not “this could be cleaner”. If a reviewer cannot point at a number, the work is accepted.

Read this together with the engineering guidelines. Where the two disagree on something specific to connectors, this document wins; on everything else, the guidelines do.

1. What you are building

A connector reads files from one storage provider — Box, pCloud, Nextcloud, a NAS, whatever the challenge names — and hands them to our service. You write the part that knows how to talk to one provider, and nothing else: the connections, the encrypted credentials, the folder selection, the synchronisation state, the ledger of what has already been seen and the OAuth dance are already written and stay ours.

There is no queue to write to. Our service holds no queue at all: it records what it found, and other services — later, on their own schedule — fetch the content and put the document through the processing pipeline. You will not see any of that from inside a connector.

1.1 A connector is a folder. One directory with a manifest, a class, its fixtures and its tests. When it is accepted, that directory is copied into our service as it is.

1.2 It touches nothing outside itself. No shared list to add yourself to, no route to register, no interface to edit. Everything the host needs to know is declared in the manifest. Work that modifies files outside its own folder is returned unread — not as a punishment, but because it means the design was worked around instead of used.

1.3 Two working examples are published. The repository github.com/vinovalab/storage-connector-example contains Google Drive and Dropbox, complete, with fixtures and green tests. Start from connectors/dropbox/: plain HTTP, path-based identifiers, a delta cursor — the shape of most providers. connectors/google-drive/ is the odd one (opaque ids, native documents that must be exported); read it for the exceptions, do not use it as a mould.

2. Before you start

2.1 The challenge is claimed on the platform. Once it is assigned to you it is yours: nobody else can take it, and you have the delivery window stated in the brief. Nothing is assigned without your application.

2.2 You need an account with the provider. We cannot give you one. A free or trial account is normally enough: it is where you try the connector against the real API and, if you record them, where the fixtures come from. If the provider has no way to obtain a developer account, say so in the application — that is a reason to change the challenge, not to work around it.

2.3 Create your repository from the template, and keep it private. Use Use this template on the example repository, not a clone: you get a clean history in a repository that is yours. It stays private, per §13.2 of the engineering guidelines — the contract and the examples are ours to publish, your connector is not yours to publish. You give us read access when you deliver.

2.4 You receive a package token when the challenge is assigned. The contract is a private package: npm install needs a credential, and it is issued to you — along with the challenge — the moment your application is accepted. Put it in an .npmrc, which the template already ignores in git:

@vinovalab:registry=https://npm.pkg.github.com
//npm.pkg.github.com/:_authToken=THE-TOKEN-YOU-RECEIVED

The token is yours, it is scoped to reading packages, and it dies with the challenge. Do not commit it, do not share it, and if you leak it by accident say so immediately: we revoke it and issue another, and nothing else happens.

2.5 After that, the whole tool-chain is npm install and npm test. No database, no Docker, no account of ours. If something in the template does not work on a clean machine, that is our defect: tell us and we fix it.

3. The shape of the folder

connectors/<name>/
  manifest.js          identity, authentication, required config, capabilities
  provider.js          the class that extends BaseProvider
  scenario.js          what the fixtures contain
  fixtures/*.json      the recorded responses
  conformance.test.js  the conformance suite plus your own tests
  README.md            what is peculiar about this provider

3.1 The manifest is how the host learns about you. Key, label, icon, authentication kind, the scopes, the environment variables you require, and what you can do — delta synchronisation, folder picker, path-based matching, native export. Nothing about your connector may be hard-coded anywhere else.

3.2 Declare every configuration key you read. The host reports a connector whose variables are missing as disabled, missing BOX_CLIENT_ID instead of silently not working. Reading a variable you did not declare defeats that, and is the one bug in this system that costs weeks to find.

3.3 The README says what is peculiar. Not what the code does — what the provider does that surprised you. The next person writing a connector reads it.

4. The contract

BaseProvider from @vinovalab/storage-connector-contract. Eight methods, and the rules the code cannot tell you.

4.1 All network traffic goes through this.http. It is an axios-shaped client the host gives you. This is what lets your connector be verified against recorded responses, without an account and without a network — which is how we review work from people whose accounts we do not have. A connector that calls the network by itself, or through a provider SDK, cannot be accepted.

4.2 downloadFile(fileId, mimeType, options) returns { buffer, mimeType }. Not a bare Buffer. The mime type of what you downloaded is not always the mime type of the file: a Google Doc comes back as a PDF, and whoever saves it needs to know.

4.3 getChanges(null) returns the cursor, not the content. The first call only fixes the starting point — { changes: [], nextPageToken, isInitial: true }. The host already does a full scan the first time; a connector that lists everything here makes it happen twice, on every connection, for ever.

4.4 A renewed token is handed back with _setCredentials(). The host then takes it with takeRefreshedCredentials() and persists it. Assigning this.credentials and nothing else leaves the connection alive until the process restarts, and dead after — with no error anywhere.

4.5 File and folder shapes are fixed. id, name, mimeType, size, checksum, modifiedAt, parentId, isIndexable, and the optional webViewLink, exportMimeType, pathLower, pathDisplay. modified instead of modifiedAt is not a detail: it produces files with no date, discovered weeks later.

4.6 If your identifiers are paths, override fileBelongsToFolder. The default compares parent identifiers. On a path-based provider that means no file inside a sub-folder is ever recognised as belonging to the monitored folder, and incremental synchronisation quietly brings back nothing. It is the most common mistake of anyone adapting an id-based connector.

4.7 downloadFile runs long after the synchronisation that stored the identifier. Not seconds later: the sync records metadata, and the bytes are fetched when some other service needs the document’s text — possibly the next day. Two things follow. The credentials have to be valid then, which is why §4.4 matters more than it looks. And the identifier you returned has to still resolve: on a path-based provider a rename or a move is enough to break it.

4.8 A file that is gone is reported as gone. When the provider says the file no longer exists, throw fileNotFound() from the contract. Every provider says it differently — Drive answers 404, Dropbox answers 409 with path/not_found, some NAS answer 200 with an error in the body — and normalising that is the connector’s job. The host reacts to fileNotFound by scheduling a re-discovery; a generic error makes it mark the document permanently broken, and an empty buffer is worse still, because it looks like a successful download of nothing and the emptiness travels into the index.

5. Conformance is the gate

5.1 npm test must be green. The conformance suite ships with the contract and is the same for everyone. It is not a formality: every check exists because that is how connectors break.

check	what it prevents
`listFiles` paginates to the end	the connector that reads the first response only: fine with ten files, loses three thousand
`getChanges(null)` returns only the cursor	re-reading the whole archive on every run
the cursor advances	seeing the same changes for ever
`downloadFile` returns `{ buffer, mimeType }`	a Google Doc saved with the wrong extension
file and folder shapes	dates that arrive null
`takeRefreshedCredentials()`	the connection that dies at token expiry
`fileBelongsToFolder` on path providers	sub-folders that never synchronise
a vanished file surfaces as `fileNotFound`	a renamed file marked permanently broken, or an empty download reaching the index
a 429 is not an empty list	the host reading “folder emptied” and deleting indexed documents

5.2 Green is necessary, not sufficient. A human reads the code. Conformance proves the shape; the review looks at what the suite cannot see — error handling, clarity, and §7.

5.3 Add your own tests. The suite checks what is the same for everyone; you know what is peculiar about your provider. The two examples each carry six or seven of their own, and they are the most useful part of both.

6. Fixtures

Fixtures are the provider’s real responses, recorded once and replayed for ever. npm run record -- <folder> drives your connector against the real API and writes them.

6.1 They must be committed. Without them nobody can run your tests: not the reviewer, not the next person, not us in six months.

6.2 They must contain no secrets. The recorder replaces the fields it knows — authorization, access_token, refresh_token, client_secret, passwords, cookies — including inside URL-encoded bodies. Read them anyway before you commit. A provider can put a token where nobody expects it, and these files end up in a repository.

6.3 The scenario says what the fixtures contain. scenario.js describes the test account: which folders, which files, which file to download and how large, which changes follow a cursor, and — when the identifiers are paths — a file in a sub-folder to prove the matching.

One of those entries is not optional: the file listing has to arrive across more than one page, so that the pagination check has something to catch. You do not need a thousand files to get there. Most providers let you ask for a small page (Drive takes pageSize, Dropbox limit), and recording with a page of two gives you the same paginated answer a full folder would give. The two published examples do exactly that.

6.4 Recorded or written by hand — but faithful. A recording from your own test account is the easiest way to be faithful, and npm run record produces one. Writing them by hand against the provider’s documented API is equally acceptable: it is what the two published examples do, because the accounts they were built from are not ours to hand around. What is never acceptable is inventing a response so that a check goes green. A reviewer who cannot reconcile a fixture with the provider’s documentation returns the work.

6.5 Never from customer data. Not a production export, not a real user’s folder, not a screenshot of one — see §13.4 of the engineering guidelines. Your own test account is not customer data; anything belonging to someone else is.

7. Security

Your connector runs inside our backend, with our customers’ OAuth tokens in memory. This section is the one where a reviewer is unforgiving.

7.1 No runtime dependencies beyond the contract. Provider SDKs are excluded by §4.1 anyway; everything else needs to be agreed before you write it. The published Google Drive connector does six HTTP calls by hand instead of pulling in a hundred megabytes of SDK.

7.2 Talk only to your provider’s hosts. Every URL your connector builds must be on the domains the manifest declares. Any other destination — telemetry, error reporting, a paste service, your own server — is a rejection, not a correction.

7.3 Read only the environment variables you declared. No process.env outside the keys in your manifest.

7.4 No filesystem, no child processes, no eval. A connector receives data and returns data. If you think you need one of these, you have found a gap in the contract: ask.

7.5 Errors keep their status. A 429 must arrive at the host as a 429 — err.status or err.response.status preserved. Swallowing an error and returning an empty list is the single most damaging thing a connector can do: the host reads it as “everything was deleted”.

8. Done means

npm test green, conformance included, on a clean checkout
fixtures committed, read, free of secrets
manifest complete: config keys, scopes, capabilities, source.repo and source.version
README that says what is peculiar about the provider
nothing modified outside connectors/<name>/
the connector works against a real account, and you say so in the delivery — what you connected, what you synchronised, what you saw

9. Delivery and review

9.1 You deliver a repository and a commit. The link to your repository and the hash of the commit you consider finished. We run npm test on that commit, read the code, and answer within the window stated in the brief.

9.2 A returned delivery cites rule numbers, and you get one round to fix them. If the fix is ours to make — an unclear rule, a defect in the template, something the contract does not cover — we make it and the work is accepted.

9.3 Acceptance includes a run against a real account. With the connector installed in our development environment. If it fails there while the fixtures are green, the fixtures did not describe reality — we work it out together, and it is not held against you.

10. Ownership and payment

10.1 The work is commissioned. On acceptance, the connector and everything in the delivery become property of Vinova Lab, which pays for it: the amount and the terms are in the brief. You keep the right to say you wrote it.

10.2 The code we publish stays published. The contract, the template and the examples are ours and are public. What you build on top of them is the work we commission.

10.3 It becomes part of a product. Your connector will run for paying customers on their own documents. That is the reason for §7 — and the reason the review is what it is.

Versions

This document is versioned. The version in force when you claim a challenge is the version that challenge is reviewed against — a change published mid-window never applies to work already claimed.

Each version is published at vinovalab.ai/work-with-us/storage-connectors with its date and hash. Substantive changes are announced before they take effect.

Version 1.0 — 30 August 2026