Subversion Repositories Spectranet

[/] [trunk/] [tnfs/] [tnfs-protocol.txt] - Rev 570

Compare with Previous | Blame | View Log

The TNFS protocol

Protocols such as NFS (unix) or SMB (Windows) are overly complex for 8 bit
systems. While NFS is well documented, it's a complex RPC based protocol.
SMB is much worse. It is also a complex RPC based protocol, but it's also
proprietary, poorly documented, and implementations differ so much that
to get something that works with a reasonable subset of SMB would add a
great deal of unwanted complexity. The Samba project has been going for 
/years/ and they still haven't finished making it bug-for-bug compatible with
the various versions of Windows!

At the other end, there's FTP, but FTP is not great for a general file
system protocol for 8 bit systems - it requires two TCP sockets for
each connection, and some things are awkward in ftp, even if they work.

So instead, TNFS provides a straightforward protocol that can easily be
implemented on most 8 bit systems. It's designed to be a bit better than
FTP for the purpose of a filesystem, but not as complex as the "big" network
filesystem protocols like NFS or SMB.

For a PC, TNFS can be implemented using something like FUSE (no, not the
Spectrum emulator, but the Filesystem In Userspace project). This is
at least available for most things Unixy (Linux, OS X, some of the BSDs),
and possibly by now for Windows also.

This is not intended to be a proper, secure network file system. If you're
storing confidential files on your Speccy, you're barmy :) Encryption,
for example, is not supported. However, servers that may be exposed to the 
internet should be coded in such a way they won't open up the host system 
to exploits.

Operations supported
These generally follow the POSIX equivalents. Entries with a * are mandatory
for servers to support.

Connection management
mount - Connect to a TNFS filesystem *
umount - Disconnect from a TNFS filesystem *

opendir - Opens a directory for reading *
readdir - Reads a directory entry *
closedir - Closes the directory *
rmdir - Removes a directory
mkdir - Creates a directory

size - Get the size of the filesystem *
free - Get the remaining free space on the filesystem *

open - Opens a file *
read - reads from an open file *
write - writes to an open file
close - closes a file *
stat - gets information about a file *
lseek - set the position in the file where the next byte will be read/written
chmod - change file access
unlink - remove a file

Note: Not all servers have to support all operations - for example, a server
on a Spectrum with a microdrive, or +3 floppy won't support
mkdir/rmdir and will only support limited options for chmod. But
a BBC Micro with ADFS would support mkdir/rmdir and more file mode options.

The directory delimiter in all cases is a "/". A server running on a filesystem
that has a different delimiter will have to translate, for example,
on a BBC with ADFS, the / would need to be translated to a "." for the
underlying OS operation. 

Protocol "on the wire"

The lowest common denominator is TNFS over UDP. UDP is the 'mandatory'
one because it demands the least of possibly small TCP/IP stacks which may
have limited resources for TCP sockets. All TNFS servers must
support the protocol on UDP port 16384. TCP is optional.

Each datagram has a header. The header is formatted the same way for all

Bytes 0,1       Connection ID (ignored for client's "mount" command)
Byte  2         Retry number
Byte  3         Command

The connection ID is to add extra identifying information, since the same
machine can establish more than one connection to the same server and may
do so with different credentials.

Byte 2 is a sequence number. This allows the receiver to determine whether
the datagram it just got is a retry or not. It should be incremented
by one for each request sent. Clients should discard datagrams from the
server if the sequence number does not match the number that was in
the request datagram.

The last byte is the command.

The remaining data in the datagram are specific to each command. However,
any command that may return more than one status (i.e. a command that
can be either succeed or fail in one or more way), byte 4 is the
status of the command, and further data follows from byte 5.

Every command should yield exactly one datagram in response. A high
level operation (such as a call to read()) asking for a buffer larger
than the size of one UDP datagram should manage this with as many requests
and responses as is necessary to fill the buffer.

The server can also ask the client to back off. If a server can operate
with interrupts enabled while the physical disc is busy, and therefore
still be able to process requests, it can tell the client that it is busy
and to try again later. In this case, the EAGAIN error code will be
returned for whatever command was being tried, and following the error
code, will be a 16 bit little endian value giving how long to back off in
milliseconds. Servers that have this ability should use it, as the server
can then better control contention on a slow device, like a floppy disc,
since the server can figure out how many requests clients are trying
to make and set the back-off value accordingly. Clients should retry as normal
once the back-off time expires.

As can be seen from this very simple wire protocol, TNFS is not designed
for confidentiality or security. You have been warned.


TNFS command datagrams

Logging on and logging off a TNFS server - MOUNT and UMOUNT commands.

MOUNT - Command ID 0x00

Standard header followed by:
Bytes 4+: 16 bit version number, little endian, LSB = minor, MSB = major
          NULL terminated string: mount location
          NULL terminated string: user id (optional - NULL if no user id)
          NULL terminated string: password (optional - NULL if no passwd)


To mount /home/tnfs on the server, with user id "example" and password of
"password", using version 1.2 of the protocol:
0x0000 0x00 0x00 0x02 0x01 /home/tnfs 0x00 example 0x00 password 0x00

To mount "a:" anonymously, using version 1.2 of the protocol:
0x0000 0x00 0x00 0x02 0x01 a: 0x00 0x00 0x00

The server responds with the standard header. If the operation was successful,
the standard header contains the session number, and the TNFS protocol
version that the server is using following the header, followed by the
minimum retry time in milliseconds as a little-endian 16 bit number.
Clients must respect this minimum retry value, especially for a server
with a slow underlying file system such as a floppy disc, to avoid swamping
the server. A client should also never have more than one request "in flight"
at any one time for any operation where order is important, so for example,
if reading a file, don't send a new request to read from a given file handle
before completing the last request.

Example: A successful MOUNT command was carried out, with a server that
supports version 2.6, and has a minimum retry time of 5 seconds (5000 ms,
hex 0x1388). Session ID is 0xBEEF:

0xBEEF 0x00 0x00 0x00 0x06 0x02 0x88 0x13

Example: A failed MOUNT command with error 1F for a version 3.5 server:
0x0000 0x00 0x00 0x1F 0x05 0x03

UMOUNT - Command ID 0x01

Standard header only, containing the connection ID to terminate, 0x00 as
the sequence number, and 0x01 as the command.

To UMOUNT the filesystem mounted with id 0xBEEF:

0xBEEF 0x00 0x01

The server responds with the standard header and a return code as byte 4.
The return code is 0x00 for OK. Example:

0xBEEF 0x00 0x01 0x00

On error, byte 4 is set to the error code, for example, for error 0x1F:

0xBEEF 0x00 0x01 0x1F

DIRECTORIES - Opening, Reading and Closing
Don't confuse this with the ability of having a directory heirachy. Even
servers (such as a +3 with a floppy) that don't have heirachical filesystems
must support cataloguing a disc, and cataloguing a disc requires opening,
reading, and closing the catalogue. It's the only way to do it!

OPENDIR - Open a directory for reading - Command ID 0x10

Standard header followed by a null terminated absolute path.
The path delimiter is always a "/". Servers whose underlying 
file system uses other delimiters, such as Acorn ADFS, should 
translate. Note that any recent version of Windows understands "/" 
to be a path delimiter, so a Windows server does not need
to translate a "/" to a "\".
Clients should keep track of their own current working directory.

0xBEEF 0x00 0x10 /home/tnfs 0x00 - Open absolute path "/home/tnfs"

The server responds with the standard header, with byte 4 set to the
return code which is 0x00 for success, and if successful, byte 5 
is set to the directory handle.

0xBEEF 0x00 0x10 0x00 0x04 - Successful, handle is 0x04
0xBEEF 0x00 0x10 0x1F - Failed with code 0x1F

READDIR - Reads a directory entry - Command ID 0x11

Standard header plus directory handle.

0xBEEF 0x00 0x11 0x04 - Read an entry with directory handle 0x04

The server responds with the standard header, followed by the directory
entry. Example:

0xBEEF 0x17 0x11 0x00 . 0x00 - Directory entry for the current working directory
0xBEEF 0x18 0x11 0x00 .. 0x00 - Directory entry for parent
0xBEEF 0x19 0x11 0x00 foo 0x00 - File named "foo"

If the end of directory is reached, or another error occurs, then the
status byte is set to the error number as for other commands.
0xBEEF 0x1A 0x11 0x21 - EOF
0xBEEF 0x1B 0x11 0x1F - Error code 0x1F

CLOSEDIR - Close a directory handle - Command ID 0x12

Standard header plus directory handle.

Example, closing handle 0x04:
0xBEEF 0x00 0x12 0x04

The server responds with the standard header, with byte 4 set to the
return code which is 0x00 for success, or something else for an error.
0xBEEF 0x00 0x12 0x00 - Close operation succeeded.
0xBEEF 0x00 0x12 0x1F - Close failed with error code 0x1F

MKDIR - Make a new directory - Command ID 0x13

Standard header plus a null-terminated absolute path.

0xBEEF 0x00 0x13 /foo/bar/baz 0x00

The server responds with the standard header plus the return code:
0xBEEF 0x00 0x13 0x00 - Directory created successfully
0xBEEF 0x00 0x13 0x02 - Directory creation failed with error 0x02

RMDIR - Remove a directory - Command ID 0x14

Standard header plus a null-terminated absolute path.

0xBEEF 0x00 0x14 /foo/bar/baz 0x00

The server responds with the standard header plus the return code:
0xBEEF 0x00 0x14 0x00 - Directory was deleted.
0xBEEF 0x00 0x14 0x02 - Directory delete operation failed with error 0x02


These typically follow the low level fcntl syscalls in Unix (and Win32),
rather than stdio and carry the same names. Note that the z88dk low level
file operations also implement these system calls. Also, some calls,
such as CREAT don't have their own packet in tnfs since they can be
implemented by something else (for example, CREAT is equivalent
to OPEN with the O_CREAT flag). Not all servers will support all flags
for OPEN, but at least O_RDONLY. The mode refers to UNIX file permissions,
see the CHMOD command below.

OPEN - Opens a file - Command 0x29
Format: Standard header, flags, mode, then the null terminated filename.
Flags are a bit field.

The flags are:
O_RDONLY        0x0001  Open read only
O_WRONLY        0x0002  Open write only
O_RDWR          0x0003  Open read/write
O_APPEND        0x0008  Append to the file, if it exists (write only)
O_CREAT         0x0100  Create the file if it doesn't exist (write only)
O_TRUNC         0x0200  Truncate the file on open for writing
O_EXCL          0x0400  With O_CREAT, returns an error if the file exists

The modes are the same as described by CHMOD (i.e. POSIX modes). These
may be modified by the server process's umask. The mode only applies
when files are created (if the O_CREAT flag is specified)

Open a file called "/foo/bar/baz.bas" for reading:

0xBEEF 0x00 0x29 0x0001 0x0000 /foo/bar/baz.bas 0x00

Open a file called "/tmp/foo.dat" for writing, creating the file but
returning an error if it exists. Modes set are S_IRUSR, S_IWUSR, S_IRGRP
and S_IWOTH (read/write for owner, read-only for group, read-only for

0xBEEF 0x00 0x29 0x0102 0x01A4 /tmp/foo.dat 0x00

The server returns the standard header and a result code in response.
If the operation was successful, the byte following the result code
is the file descriptor:

0xBEEF 0x00 0x29 0x00 0x04 - Successful file open, file descriptor = 4
0xBEEF 0x00 0x29 0x01 - File open failed with "permssion denied"

(HISTORICAL NOTE: OPEN used to have command id 0x20, but with the
addition of extra flags, the id was changed so that servers could
support both the old style OPEN and the new OPEN)

READ - Reads from a file - Command 0x21
Reads a block of data from a file. Consists of the standard header
followed by the file descriptor as returned by OPEN, then a 16 bit
little endian integer specifying the size of data that is requested.

The server will only reply with as much data as fits in the maximum
TNFS datagram size of 1K when using UDP as a transport. For the
TCP transport, sequencing and buffering etc. are just left up to
the TCP stack, so a READ operation can return blocks of up to 64K. 

If there is less than the size requested remaining in the file, 
the server will return the remainder of the file.  Subsequent READ 
commands will return the code EOF.

Read from fd 4, maximum 256 bytes:

0xBEEF 0x00 0x21 0x04 0x00 0x01

The server will reply with the standard header, followed by the single
byte return code, the actual amount of bytes read as a 16 bit unsigned
little endian value, then the data, for example, 256 bytes:

0xBEEF 0x00 0x21 0x00 0x00 0x01

End-of-file reached:

0xBEEF 0x00 0x21 0x21

WRITE - Writes to a file - Command 0x22
Writes a block of data to a file. Consists of the standard header,
followed by the file descriptor, followed by a 16 bit little endian
value containing the size of the data, followed by the data. The
entire message must fit in a single datagram.

Write to fd 4, 256 bytes of data:

0xBEEF 0x00 0x22 0x04 0x00 0x01

The server replies with the standard header, followed by the return
code, and the number of bytes actually written. For example:

0xBEEF 0x00 0x22 0x00 0x00 0x01 - Successful write of 256 bytes
0xBEEF 0x00 0x22 0x06 - Failed write, error is "bad file descriptor"

CLOSE - Closes a file - Command 0x23
Closes an open file. Consists of the standard header, followed by
the file descriptor. Example:

0xBEEF 0x00 0x23 0x04 - Close file descriptor 4

The server replies with the standard header followed by the return

0xBEEF 0x00 0x23 0x00 - File closed.
0xBEEF 0x00 0x23 0x06 - Operation failed with EBADF, "bad file descriptor"

STAT - Get information on a file - Command 0x24
Reads the file's information, such as size, datestamp etc. The TNFS
stat contains less data than the POSIX stat - information that is unlikely
to be of use to 8 bit systems are omitted.
The request consists of the standard header, followed by the full path
of the file to stat, terminated by a NULL. Example:

0xBEEF 0x00 0x24 /foo/bar/baz.txt 0x00

The server replies with the standard header, followed by the return code.
On success, the file information follows this. Stat information is returned
in this order. Not all values are used by all servers. At least file
mode and size must be set to a valid value (many programs depend on these).

File mode       - 2 bytes: file permissions - little endian byte order
uid             - 2 bytes: Numeric UID of owner
gid             - 2 bytes: Numeric GID of owner
size            - 4 bytes: Unsigned 32 bit little endian size of file in bytes
atime           - 4 bytes: Access time in seconds since the epoch, little end.
mtime           - 4 bytes: Modification time in seconds since the epoch,
                           little endian
ctime           - 4 bytes: Time of last status change, as above.
uidstring       - 0 or more bytes: Null terminated user id string
gidstring       - 0 or more bytes: Null terminated group id string

Fields that don't apply to the server in question should be left as 0x00.
The ┬┤mtime' field and 'size' fields are unsigned 32 bit integers.
The uidstring and gidstring are helper fields so the client doesn't have
to then ask the server for the string representing the uid and gid.

File mode flags will be most useful for code that is showing a directory
listing, and for programs that need to find out what kind of file (regular
file or directory, etc) a particular file may be. They follow the POSIX
convention which is:

Flags           Octal representation
S_IFMT          0170000         Bitmask for the file type bitfields
S_IFSOCK        0140000         Is a socket
S_IFLNK         0120000         Is a symlink
S_IFREG         0100000         Is a regular file
S_IFBLK         0060000         block device
S_IFDIR         0040000         Directory
S_IFCHR         0020000         Character device
S_IFIFO         0010000         FIFO
S_ISUID         0004000         set UID bit
S_ISGID         0002000         set group ID bit
S_ISVTX         0001000         sticky bit
S_IRWXU         00700           Mask for file owner permissions
S_IRUSR         00400           owner has read permission
S_IWUSR         00200           owner has write permission
S_IXUSR         00100           owner has execute permission
S_IRGRP         00040           group has read permission
S_IWGRP         00020           group has write permission
S_IXGRP         00010           group has execute permission
S_IROTH         00004           others have read permission
S_IWOTH         00002           others have write permission
S_IXOTH         00001           others have execute permission

Most of these won't be of much interest to an 8 bit client, but the
read/write/execute permissions can be used for a client to determine whether
to bother even trying to open a remote file, or to automatically execute
certain types of files etc. (Further file metadata such as load and execution
addresses are platform specific and should go into a header of the file
in question). Note the "trivial" bit in TNFS means that the client is
unlikely to do anything special with a FIFO, so writing to a file of that
type is likely to have effects on the server, and not the client! It's also
worth noting that the server is responsible for enforcing read and write
permissions (although the permission bits can help the client work out
whether it should bother to send a request).

LSEEK - Seeks to a new position in a file - Command 0x25
Seeks to an absolute position in a file, or a relative offset in a file,
or to the end of a file.
The request consists of the header, followed by the file descriptor,
followed by the seek type (SEEK_SET, SEEK_CUR or SEEK_END), followed
by the position to seek to. The seek position is a signed 32 bit integer,
little endian. (2GB file sizes should be more than enough for 8 bit

The seek types are defined as follows:
0x00            SEEK_SET - Go to an absolute position in the file
0x01            SEEK_CUR - Go to a relative offset from the current position
0x02            SEEK_END - Seek to EOF


File descriptor is 4, type is SEEK_SET, and position is 0xDEADBEEF:
0xBEEF 0x00 0x25 0x04 0x00 0xEF 0xBE 0xAD 0xDE

Note that clients that buffer reads for single-byte reads will have
to make a calculation to implement SEEK_CUR correctly since the server's
file pointer will be wherever the last read block made it end up.

UNLINK - Unlinks (deletes) a file - Command 0x26
Removes the specified file. The request consists of the header then
the null terminated full path to the file. The reply consists of the
header and the return code.

Unlink file "/foo/bar/baz.bas"
0xBEEF 0x00 0x26 /foo/bar/baz.bas 0x00

CHMOD - Changes permissions on a file - Command 0x27
Changes file permissions on the specified file, using POSIX permissions
semantics. Not all permissions may be supported by all servers - most 8
bit systems, for example, may only support removing the write bit.
A server running on something Unixish will support everything.
The request consists of the header, followed by the 16 bit file mode,
followed by the null terminated filename. Filemode is sent as a little
endian value. See the Unix manpage for chmod(2) for further information.

File modes are as defined by POSIX. The POSIX definitions are as follows:
Flag      Octal Description
S_ISUID   04000 set user ID on execution
S_ISGID   02000 set group ID on execution
S_ISVTX   01000 sticky bit
S_IRUSR   00400 read by owner
S_IWUSR   00200 write by owner
S_IXUSR   00100 execute/search by owner
S_IRGRP   00040 read by group
S_IWGRP   00020 write by group
S_IXGRP   00010 execute/search by group
S_IROTH   00004 read by others
S_IWOTH   00002 write by others
S_IXOTH   00001 execute/search by others

Example: Set permissions to 755 on /foo/bar/baz.bas:
0xBEEF 0x00 0x27 0xED 0x01 /foo/bar/baz.bas

The reply is the standard header plus the return code of the chmod operation.

RENAME - Moves a file within a filesystem - Command 0x28
Renames a file (or moves a file within a filesystem - it must be possible
to move a file to a different directory within the same FS on the
server using this command).
The request consists of the header, followed by the null terminated
source path, and the null terminated destination path.

Example: Move file "foo.txt" to "bar.txt"
0xBEEF 0x00 0x28 foo.txt 0x00 bar.txt 0x00

These operations get information about the device that is mounted.

SIZE - Requests the size of the mounted filesystem - Command 0x30
Finds the size, in kilobytes, of the filesystem that is currently mounted.
The request consists of a standard header and nothing more.

0xBEEF 0x00 0x30

The reply is the standard header, followed by the return code, followed
by a 32 bit little endian integer which is the size of the filesystem
in kilobytes, for example:

0xBEEF 0x00 0x30 0x00 0xD0 0x02 0x00 0x00 - Filesystem is 720kbytes
0xBEEF 0x00 0x30 0xFF - Request failed with error code 0xFF

FREE - Requests the amount of free space on the filesystem - Command 0x31
Finds the size, in kilobytes, of the free space remaining on the mounted
filesystem. The request consists of the standard header and nothing more.

0xBEEF 0x00 0x31

The reply is as for SIZE - the standard header, return code, and little
endian integer for the free space in kilobytes, for example:

0xBEEF 0x00 0x31 0x00 0x64 0x00 0x00 0x00 - There is 64K free.
0xBEEF 0x00 0x31 0x1F - Request failed with error 0x1F

Note not all servers may return all codes. For example, a server on a machine
that doesn't have named pipes will never return ESPIPE.

ID      POSIX equiv     Description
0x00                    Success
0x01    EPERM           Operation not permitted
0x02    ENOENT          No such file or directory
0x03    EIO             I/O error               
0x04    ENXIO           No such device or address
0x05    E2BIG           Argument list too long
0x06    EBADF           Bad file number
0x07    EAGAIN          Try again
0x08    ENOMEM          Out of memory
0x09    EACCES          Permission denied
0x0A    EBUSY           Device or resource busy
0x0B    EEXIST          File exists
0x0C    ENOTDIR         Is not a directory
0x0D    EISDIR          Is a directory
0x0E    EINVAL          Invalid argument
0x0F    ENFILE          File table overflow
0x10    EMFILE          Too many open files
0x11    EFBIG           File too large
0x12    ENOSPC          No space left on device
0x13    ESPIPE          Attempt to seek on a FIFO or pipe
0x14    EROFS           Read only filesystem
0x15    ENAMETOOLONG    Filename too long
0x16    ENOSYS          Function not implemented
0x17    ENOTEMPTY       Directory not empty
0x18    ELOOP           Too many symbolic links encountered
0x19    ENODATA         No data available
0x1A    ENOSTR          Out of streams resources
0x1B    EPROTO          Protocol error
0x1C    EBADFD          File descriptor in bad state
0x1D    EUSERS          Too many users
0x1E    ENOBUFS         No buffer space available
0x1F    EALREADY        Operation already in progress
0x20    ESTALE          Stale TNFS handle
0x21    EOF             End of file
0xFF                    Invalid TNFS handle

Compare with Previous | Blame | View Log