SFTP access

Any SFTP client can reach your WorkData: command-line sftp, GUI clients such as WinSCP, Cyberduck or FileZilla, and scripts using rclone, lftp or Python. Folders appear as directories, and WorkData appear as files.

Connecting

Host	Port	Host key fingerprint
jobs.api.pinexq.net	2222	SHA256:Yfd7W+Q5xakCDNOyWOrVs+gNx9SqYUVjfsyy6ZuD6dE

SFTP uses the same host as the API, on port 2222.

The first time you connect, your client shows the server’s host key fingerprint. Compare it with the fingerprint above before accepting it. If a client later warns that the host key has changed, stop and contact us. Don’t accept the new key.

Logging in

Log in with one of the following:

• An API key as the password. Create one in the portal under Settings → API keys, as described in Getting started.
• An SSH key. In the portal, go to Settings → SSH Keys and click Add SSH Key. You can upload an existing public key, or let PineXQ generate a key pair. The generated private key is shown only once: save it as ~/.ssh/id_ed25519_pinexq and run chmod 600 ~/.ssh/id_ed25519_pinexq.

The username is ignored. Your key alone determines who you are, but clients still need some username, so enter anything, for example pinexq or your organisation name.

Note

A key created while an organization is selected in the portal shows that organization’s WorkData. A key created without one shows your personal WorkData. One connection sees one or the other. To switch, log in with a different key.

Every API key works for SFTP, with the same rights it has for the API. Treat an API key you use for SFTP like any other API key, and delete it in the portal when it is no longer needed.

Client setup

sftp (OpenSSH)

With an API key, enter it at the password prompt:

sftp -P 2222 -o PubkeyAuthentication=no pinexq@jobs.api.pinexq.net

PubkeyAuthentication=no skips the keys in your SSH agent and goes straight to the password prompt.

With an SSH key:

sftp -P 2222 -i ~/.ssh/id_ed25519_pinexq pinexq@jobs.api.pinexq.net

For scripts, put the commands in a file and run sftp -b commands.txt … with an SSH key.

WinSCP

1. New site, File protocol: SFTP.
2. Host name: jobs.api.pinexq.net, Port number: 2222.
3. User name: anything, for example pinexq.
4. Password: your API key. Or, for an SSH key: Advanced… → SSH → Authentication → Private key file. WinSCP converts an OpenSSH key to its own format when asked.

Cyberduck

1. Open Connection, choose SFTP (SSH File Transfer Protocol).
2. Server: jobs.api.pinexq.net, Port: 2222.
3. Username: anything, for example pinexq.
4. Password: your API key, or select your private key under SSH Private Key.

FileZilla

1. File → Site Manager → New site, Protocol: SFTP - SSH File Transfer Protocol.
2. Host: jobs.api.pinexq.net, Port: 2222.
3. Logon Type: Normal with your API key as the password, or Key file with your private key.
4. User: anything, for example pinexq.

rclone

rclone config create pinexq sftp \
    host=jobs.api.pinexq.net port=2222 user=pinexq \
    pass="$PINEXQ_API_KEY" shell_type=none disable_hashcheck=true --obscure

rclone ls pinexq:
rclone copy --inplace --size-only ./results pinexq:results

• shell_type=none and disable_hashcheck=true: the server runs no shell commands, so rclone shouldn’t try any.
• --inplace: upload straight to the final name. Without it, rclone uploads to a temporary .partial name first.
• --size-only: compare files by size. Modification times can’t be set (see below), so comparing times would upload every file again.

For an SSH key, use key_file=~/.ssh/id_ed25519_pinexq instead of pass.

lftp

lftp -p 2222 -u pinexq sftp://jobs.api.pinexq.net

lftp asks for the password; enter your API key. To upload a local directory:

mirror --reverse --ignore-time --no-perms ./results results

• --ignore-time: modification times can’t be set.
• --no-perms: by default mirror copies file permissions. On this server, permissions decide whether a file can be deleted, so copying them could make whole trees deletable.

Python (paramiko)

import os
import paramiko

client = paramiko.SSHClient()
client.load_system_host_keys()  # accept the host key once with sftp, or add it to known_hosts
client.connect(
    'jobs.api.pinexq.net',
    port=2222,
    username='pinexq',
    password=os.environ['PINEXQ_API_KEY'],
    allow_agent=False,
    look_for_keys=False,
)
sftp = client.open_sftp()
print(sftp.listdir('.'))
sftp.put('report.csv', 'reports/report.csv')
client.close()

Don’t use paramiko.AutoAddPolicy(): it accepts any host key without checking it.

Differences from a regular file server

WorkData are immutable and carry less metadata than files, so some operations behave differently.

• No overwrite, append or resume. Uploading to a name that already exists fails. Delete the old file first, then upload. An interrupted upload has to start again from the beginning.

• Deleting needs a chmod. A WorkData can only be deleted when it is marked deletable, which is off by default, also for files you upload. The file’s permissions show it: -r--r--r-- (0444) is protected, and -rw-r--r-- (0644) can be deleted.

  sftp> chmod 644 old-report.csv
  sftp> rm old-report.csv

  chmod 444 protects a file again. WorkData used as input of a job or template can’t be deleted at all. Their chmod 644 and rm return “Permission denied”.

• Times. Files show the time the WorkData was created. A file’s modification time can’t be changed: clients that set it (for example put -p, or WinSCP’s “preserve timestamp”) report success, but nothing changes.

• Duplicate names. WorkData names don’t have to be unique, but file names do. When WorkData in a folder share a name, the newest keeps it, and each older one is listed with part of its id added, for example report~3f2a91c0.csv. So report.csv is always the newest: a newer report.csv uploaded through the portal takes the name over, and deleting report.csv gives it to the next newest.

• / in names. A WorkData whose name contains / is listed with ∕ (a look-alike character) in its place.

• Not everything is listed. Hidden WorkData are left out, and so are jobs and other non-WorkData items of a folder. Secret WorkData are listed but can’t be downloaded.

• Upload size. At most 5 GB per file.

• Folders. mkdir, rmdir (empty folders only) and rename work as usual. Recursive deletes are your client’s job, and each file still needs to be deletable.

Limits

• Idle connections are closed after 15 minutes without traffic. Every connection ends 8 hours after login, even while transferring.
• Up to 10 connections per user at once. Each connection may run 4 uploads at once.
• Wrong passwords: a connection gets 5 password attempts. After many failed logins from the same address, that address is blocked for 10 minutes.
• Very large folders: a folder with more than 250,000 items can’t be listed. Its subfolders still work.
• Revoked keys: a deleted API key or SSH key can no longer log in. Connections that are already open keep working until they end, at most 8 hours after login.