Subversion Repositories Spectranet

[/] [trunk/] [tnfs/] [tnfs-protocol.txt] - Rev 188

Go to most recent revision | Compare with Previous | Blame | View Log

The TNFS protocol

Protocols such as NFS (unix) or SMB (Windows) are overly complex for 8 bit
systems. While NFS is well documented, it's a complex RPC based protocol.
SMB is much worse. It is also a complex RPC based protocol, but it's also
proprietary, poorly documented, and implementations differ so much that
to get something that works with a reasonable subset of SMB would add a
great deal of unwanted complexity. The Samba project has been going for 
/years/ and they still haven't finished making it bug-for-bug compatible with
the various versions of Windows!

At the other end, there's FTP, but FTP is not great for a general file
system protocol for 8 bit systems - it requires two TCP sockets for
each connection, and some things are awkward in ftp, even if they work.

So instead, TNFS provides a straightforward protocol that can easily be
implemented on most 8 bit systems. It's designed to be a bit better than
FTP for the purpose of a filesystem, but not as complex as the "big" network
filesystem protocols like NFS or SMB.

For a PC, TNFS can be implemented using something like FUSE (no, not the
Spectrum emulator, but the Filesystem In Userspace project). This is
at least available for most things Unixy (Linux, OS X, some of the BSDs),
and possibly by now for Windows also.

This is not intended to be a proper, secure network file system. If you're
storing confidential files on your Speccy, you're barmy :) Encryption,
for example, is not supported. However, servers that may be exposed to the 
internet should be coded in such a way they won't open up the host system 
to exploits.

Operations supported
These generally follow the POSIX equivalents. Entries with a * are mandatory
for servers to support.

Connection management
mount - Connect to a TNFS filesystem *
umount - Disconnect from a TNFS filesystem *

opendir - Opens a directory for reading *
readdir - Reads a directory entry *
closedir - Closes the directory *
rmdir - Removes a directory
mkdir - Creates a directory

size - Get the size of the filesystem *
free - Get the remaining free space on the filesystem *

open - Opens a file *
read - reads from an open file *
write - writes to an open file
close - closes a file *
stat - gets information about a file *
lseek - set the position in the file where the next byte will be read/written
chmod - change file access
unlink - remove a file

Note: Not all servers have to support all operations - for example, a server
on a Spectrum with a microdrive, or +3 floppy won't support
chdir/mkdir/rmdir and will only support limited options for chmod. But
a BBC Micro with ADFS would support chdir/mkdir/readdir.

The directory delimiter in all cases is a "/". A server running on a filesystem
that has a different delimiter will have to translate, for example,
on a BBC with ADFS, the / would need to be translated to a "." for the
underlying OS operation. 

Protocol "on the wire"

The lowest common denominator is TNFS over UDP. UDP is the 'mandatory'
one because it demands the least of possibly small TCP/IP stacks which may
have limited resources for TCP sockets. All TNFS servers must
support the protocol on UDP port 16384. TCP is optional.

Each datagram has a header. The header is formatted the same way for all

Bytes 0,1       Connection ID (ignored for client's "mount" command)
Byte  2         Retry number
Byte  3         Command

The connection ID is to add extra identifying information, since the same
machine can establish more than one connection to the same server and may
do so with different credentials.

Byte 2 is a retry number. This allows the receiver to determine whether
the datagram it just got is a retry or not. When byte 2 is set to 0x00,
this is the first attempt at sending the command. For example, if a server
receives a read command with retry set to 0x01, it knows it should resend
the last block instead of sending a new one.

The last byte is the command.

The remaining data in the datagram are specific to each command. However,
any command that may return more than one status (i.e. a command that
can be either succeed or fail in one or more way), byte 4 is the
status of the command, and further data follows from byte 5.

Every command should yield exactly one datagram in response. A high
level operation (such as a call to read()) asking for a buffer larger
than the size of one UDP datagram should manage this with as many requests
and responses as is necessary to fill the buffer.

The server can also ask the client to back off. If a server can operate
with interrupts enabled while the physical disc is busy, and therefore
still be able to process requests, it can tell the client that it is busy
and to try again later. In this case, the EAGAIN error code will be
returned for whatever command was being tried, and following the error
code, will be a 16 bit little endian value giving how long to back off in
milliseconds. Servers that have this ability should use it, as the server
can then better control contention on a slow device, like a floppy disc,
since the server can figure out how many requests clients are trying
to make and set the back-off value accordingly. Clients should retry as normal
once the back-off time expires.

As can be seen from this very simple wire protocol, TNFS is not designed
for confidentiality.


TNFS command datagrams

Logging on and logging off a TNFS server - MOUNT and UMOUNT commands.

MOUNT - Command ID 0x00

Standard header followed by:
Bytes 4+: 16 bit version number, little endian, LSB = minor, MSB = major
          NULL terminated string: mount location
          NULL terminated string: user id (optional - NULL if no user id)
          NULL terminated string: password (optional - NULL if no passwd)


To mount /home/tnfs on the server, with user id "example" and password of
"password", using version 1.2 of the protocol:
0x0000 0x00 0x00 0x02 0x01 /home/tnfs 0x00 example 0x00 password 0x00

To mount "a:" anonymously, using version 1.2 of the protocol:
0x0000 0x00 0x00 0x02 0x01 a: 0x00 0x00 0x00

The server responds with the standard header. If the operation was successful,
the standard header contains the session number, and the TNFS protocol
version that the server is using following the header, followed by the
minimum retry time in milliseconds as a little-endian 16 bit number.
Clients must respect this minimum retry value, especially for a server
with a slow underlying file system such as a floppy disc, to avoid swamping
the server. A client should also never have more than one request "in flight"
at any one time for any operation where order is important, so for example,
if reading a file, don't send a new request to read from a given file handle
before completing the last request.

Example: A successful MOUNT command was carried out, with a server that
supports version 2.6, and has a minimum retry time of 5 seconds (5000 ms,
hex 0x1388). Session ID is 0xBEEF:

0xBEEF 0x00 0x00 0x00 0x06 0x02 0x88 0x13

Example: A failed MOUNT command with error 1F for a version 3.5 server:
0x0000 0x00 0x00 0x1F 0x05 0x03

UMOUNT - Command ID 0x01

Standard header only, containing the connection ID to terminate, 0x00 as
the sequence number, and 0x01 as the command.

To UMOUNT the filesystem mounted with id 0xBEEF:

0xBEEF 0x00 0x01

The server responds with the standard header and a return code as byte 4.
The return code is 0x00 for OK. Example:

0xBEEF 0x00 0x01 0x00

On error, byte 4 is set to the error code, for example, for error 0x1F:

0xBEEF 0x00 0x01 0x1F

DIRECTORIES - Opening, Reading and Closing
Don't confuse this with the ability of having a directory heirachy. Even
servers (such as a +3 with a floppy) that don't have heirachical filesystems
must support cataloguing a disc, and cataloguing a disc requires opening,
reading, and closing the catalogue. It's the only way to do it!

OPENDIR - Open a directory for reading - Command ID 0x10

Standard header followed by a null terminated absolute path.
The path delimiter is always a "/". Servers whose underlying 
file system uses other delimiters, such as Acorn ADFS, should 
translate. Note that any recent version of Windows understands "/" 
to be a path delimiter, so a Windows server does not need
to translate a "/" to a "\".
Clients should keep track of their own current working directory.

0xBEEF 0x00 0x10 /home/tnfs 0x00 - Open absolute path "/home/tnfs"

The server responds with the standard header, with byte 4 set to the
return code which is 0x00 for success, and if successful, byte 5 
is set to the directory handle.

0xBEEF 0x00 0x10 0x00 0x04 - Successful, handle is 0x04
0xBEEF 0x00 0x10 0x1F - Failed with code 0x1F

READDIR - Reads a directory entry - Command ID 0x11

Standard header plus directory handle.

0xBEEF 0x00 0x11 0x04 - Read an entry with directory handle 0x04

The server responds with the standard header, with the sequence number
set to the directory entry number. Following the standard header is
a NULL terminated string, which is the directory entry. Example:
0xBEEF 0x00 0x11 . 0x00 - Directory entry for the current working directory
0xBEEF 0x00 0x11 .. 0x00 - Directory entry for parent
0xBEEF 0x00 0x11 foo 0x00 - File named "foo"

If the end of directory is reached, or other error is reached, the
header is sent as above, but byte 4 is set to 0x00 and byte 5 is set to
the code (either EOF or an error):
0xBEEF 0x00 0x11 0x00 0x01 - EOF
0xBEEF 0x00 0x11 0x00 0x1F - Error code 0x1F

CLOSEDIR - Close a directory handle - Command ID 0x12

Standard header plus directory handle.

Example, closing handle 0x04:
0xBEEF 0x00 0x12 0x04

The server responds with the standard header, with byte 4 set to the
return code which is 0x00 for success, or something else for an error.
0xBEEF 0x00 0x12 0x00 - Close operation succeeded.
0xBEEF 0x00 0x12 0x1F - Close failed with error code 0x1F


These typically follow the low level fcntl syscalls in Unix (and Win32),
rather than stdio and carry the same names. Note that the z88dk low level
file operations also implement these system calls. Also, some calls,
such as CREAT don't have their own packet in tnfs since they can be
implemented by something else (for example, CREAT is equivalent
to OPEN with the O_CREAT flag). Not all servers will support all flags
for OPEN, but at least O_RDONLY

OPEN - Opens a file - Command 0x20
Format: Standard header, flags, filemode, then the null terminated filename.
Flags are a bit field.
The filemodes are:

O_RDONLY        0x01    Open read only
O_WRONLY        0x02    Open write only
O_RDWR          0x03    Open read/write

The flags are:

O_APPEND        0x01    Append to the file, if it exists (write only)
O_CREAT         0x02    Create the file if it doesn't exist (write only)
O_EXCL          0x04    With O_CREAT, returns an error if the file exists
O_TRUNC         0x08    Truncate the file on open for writing

Open a file called "/foo/bar/baz.bas" for reading:

0xBEEF 0x00 0x20 0x01 0x00 /foo/bar/baz.bas 0x00

Open a file called "/tmp/foo.dat" for writing, creating the file but
returning an error if it exists:

0xBEEF 0x00 0x20 0x02 0x06 /tmp/foo.dat 0x00

The server returns the standard header and a result code in response.
If the operation was successful, the byte following the result code
is the file descriptor:

0xBEEF 0x00 0x20 0x00 0x04 - Successful file open, file descriptor = 4
0xBEEF 0x00 0x20 0x01 - File open failed with "permssion denied"

READ - Reads from a file - Command 0x21
Reads a block of data from a file. Consists of the standard header
followed by the file descriptor as returned by OPEN, then a 16 bit
little endian integer specifying the size of data that is requested.
The server will only reply with as much data as fits in the maximum
TNFS datagram size of 1K. If there is less than the size requested
remaining in the file, the server will return the remainder of the file.
Subsequent READ commands will return the code EOF.

Read from fd 4, maximum 256 bytes:

0xBEEF 0x00 0x21 0x04 0x00 0x01

The server will reply with the standard header, followed by the single
byte return code, the actual amount of bytes read as a 16 bit unsigned
little endian value, then the data, for example, 256 bytes:

0xBEEF 0x00 0x21 0x00 0x00 0x01

End-of-file reached:

0xBEEF 0x00 0x21 0x21

WRITE - Writes to a file - Command 0x22
Writes a block of data to a file. Consists of the standard header,
followed by the file descriptor, followed by a 16 bit little endian
value containing the size of the data, followed by the data. The
entire message must fit in a single datagram.

Write to fd 4, 256 bytes of data:

0xBEEF 0x00 0x22 0x04 0x00 0x01

The server replies with the standard header, followed by the return
code, and the number of bytes actually written. For example:

0xBEEF 0x00 0x22 0x00 0x00 0x01 - Successful write of 256 bytes
0xBEEF 0x00 0x22 0x06 - Failed write, error is "bad file descriptor"

CLOSE - Closes a file - Command 0x23
Closes an open file. Consists of the standard header, followed by
the file descriptor. Example:

0xBEEF 0x00 0x23 0x04 - Close file descriptor 4

The server replies with the standard header followed by the return

0xBEEF 0x00 0x23 0x00 - File closed.
0xBEEF 0x00 0x23 0x06 - Operation failed with EBADF, "bad file descriptor"

Note not all servers may return all codes. For example, a server on a machine
that doesn't have named pipes will never return ESPIPE.

ID      POSIX equiv     Description
0x00                    Success
0x01    EPERM           Operation not permitted
0x02    ENOENT          No such file or directory
0x03    EIO             I/O error               
0x04    ENXIO           No such device or address
0x05    E2BIG           Argument list too long
0x06    EBADF           Bad file number
0x07    EAGAIN          Try again
0x08    ENOMEM          Out of memory
0x09    EACCES          Permission denied
0x0A    EBUSY           Device or resource busy
0x0B    EEXIST          File exists
0x0C    ENOTDIR         Is not a directory
0x0D    EISDIR          Is a directory
0x0E    EINVAL          Invalid argument
0x0F    ENFILE          File table overflow
0x10    EMFILE          Too many open files
0x11    EFBIG           File too large
0x12    ENOSPC          No space left on device
0x13    ESPIPE          Attempt to seek on a FIFO or pipe
0x14    EROFS           Read only filesystem
0x15    ENAMETOOLONG    Filename too long
0x16    ENOSYS          Function not implemented
0x17    ENOTEMPTY       Directory not empty
0x18    ELOOP           Too many symbolic links encountered
0x19    ENODATA         No data available
0x1A    ENOSTR          Out of streams resources
0x1B    EPROTO          Protocol error
0x1C    EBADFD          File descriptor in bad state
0x1D    EUSERS          Too many users
0x1E    ENOBUFS         No buffer space available
0x1F    EALREADY        Operation already in progress
0x20    ESTALE          Stale TNFS handle
0x21    EOF             End of file
0xFF                    Invalid TNFS handle

Go to most recent revision | Compare with Previous | Blame | View Log