Subversion Repositories Spectranet

[/] [trunk/] [tnfs/] [tnfs-protocol.txt] - Blame information for rev 197

Go to most recent revision | Details | Compare with Previous | View Log

Line No. Rev Author Line
1 178 winston
The TNFS protocol
2
=================
3
 
4
Rationale
5
---------
6
Protocols such as NFS (unix) or SMB (Windows) are overly complex for 8 bit
7
systems. While NFS is well documented, it's a complex RPC based protocol.
8
SMB is much worse. It is also a complex RPC based protocol, but it's also
9
proprietary, poorly documented, and implementations differ so much that
10
to get something that works with a reasonable subset of SMB would add a
11
great deal of unwanted complexity. The Samba project has been going for
12
/years/ and they still haven't finished making it bug-for-bug compatible with
13
the various versions of Windows!
14
 
15
At the other end, there's FTP, but FTP is not great for a general file
16
system protocol for 8 bit systems - it requires two TCP sockets for
17
each connection, and some things are awkward in ftp, even if they work.
18
 
19
So instead, TNFS provides a straightforward protocol that can easily be
20
implemented on most 8 bit systems. It's designed to be a bit better than
21
FTP for the purpose of a filesystem, but not as complex as the "big" network
22
filesystem protocols like NFS or SMB.
23
 
24
For a PC, TNFS can be implemented using something like FUSE (no, not the
25
Spectrum emulator, but the Filesystem In Userspace project). This is
26
at least available for most things Unixy (Linux, OS X, some of the BSDs),
27
and possibly by now for Windows also.
28
 
29
Security
30
--------
31
This is not intended to be a proper, secure network file system. If you're
32
storing confidential files on your Speccy, you're barmy :) Encryption,
33
for example, is not supported. However, servers that may be exposed to the
34
internet should be coded in such a way they won't open up the host system
35
to exploits.
36
 
37
Operations supported
38
====================
39
These generally follow the POSIX equivalents. Entries with a * are mandatory
40
for servers to support.
41
 
42
Connection management
43
---------------------
44
mount - Connect to a TNFS filesystem *
45
umount - Disconnect from a TNFS filesystem *
46
 
47
Directories
48
-----------
49
opendir - Opens a directory for reading *
50
readdir - Reads a directory entry *
51
closedir - Closes the directory *
52
rmdir - Removes a directory
53
mkdir - Creates a directory
54
 
55 182 winston
Devices
56
-------
57
size - Get the size of the filesystem *
58
free - Get the remaining free space on the filesystem *
59
 
60 178 winston
Files
61
-----
62
open - Opens a file *
63
read - reads from an open file *
64
write - writes to an open file
65
close - closes a file *
66
stat - gets information about a file *
67
lseek - set the position in the file where the next byte will be read/written
68
chmod - change file access
69
unlink - remove a file
70
 
71
Note: Not all servers have to support all operations - for example, a server
72
on a Spectrum with a microdrive, or +3 floppy won't support
73 197 winston
mkdir/rmdir and will only support limited options for chmod. But
74
a BBC Micro with ADFS would support mkdir/rmdir and more file mode options.
75 178 winston
 
76
The directory delimiter in all cases is a "/". A server running on a filesystem
77
that has a different delimiter will have to translate, for example,
78
on a BBC with ADFS, the / would need to be translated to a "." for the
79
underlying OS operation.
80
 
81
Protocol "on the wire"
82
======================
83
 
84
The lowest common denominator is TNFS over UDP. UDP is the 'mandatory'
85
one because it demands the least of possibly small TCP/IP stacks which may
86
have limited resources for TCP sockets. All TNFS servers must
87
support the protocol on UDP port 16384. TCP is optional.
88
 
89
Each datagram has a header. The header is formatted the same way for all
90
datagrams.
91
 
92
Bytes 0,1       Connection ID (ignored for client's "mount" command)
93 182 winston
Byte  2         Retry number
94 178 winston
Byte  3         Command
95
 
96 182 winston
The connection ID is to add extra identifying information, since the same
97
machine can establish more than one connection to the same server and may
98
do so with different credentials.
99 178 winston
 
100 182 winston
Byte 2 is a retry number. This allows the receiver to determine whether
101
the datagram it just got is a retry or not. When byte 2 is set to 0x00,
102
this is the first attempt at sending the command. For example, if a server
103
receives a read command with retry set to 0x01, it knows it should resend
104
the last block instead of sending a new one.
105 178 winston
 
106
The last byte is the command.
107
 
108 182 winston
The remaining data in the datagram are specific to each command. However,
109
any command that may return more than one status (i.e. a command that
110
can be either succeed or fail in one or more way), byte 4 is the
111
status of the command, and further data follows from byte 5.
112 178 winston
 
113 182 winston
Every command should yield exactly one datagram in response. A high
114
level operation (such as a call to read()) asking for a buffer larger
115
than the size of one UDP datagram should manage this with as many requests
116
and responses as is necessary to fill the buffer.
117
 
118
The server can also ask the client to back off. If a server can operate
119
with interrupts enabled while the physical disc is busy, and therefore
120
still be able to process requests, it can tell the client that it is busy
121
and to try again later. In this case, the EAGAIN error code will be
122
returned for whatever command was being tried, and following the error
123
code, will be a 16 bit little endian value giving how long to back off in
124
milliseconds. Servers that have this ability should use it, as the server
125
can then better control contention on a slow device, like a floppy disc,
126
since the server can figure out how many requests clients are trying
127
to make and set the back-off value accordingly. Clients should retry as normal
128
once the back-off time expires.
129
 
130 178 winston
As can be seen from this very simple wire protocol, TNFS is not designed
131
for confidentiality.
132
 
133 182 winston
------------------------------------------------------------------------------
134
 
135 178 winston
TNFS command datagrams
136
======================
137
 
138
Logging on and logging off a TNFS server - MOUNT and UMOUNT commands.
139
---------------------------------------------------------------------
140
 
141
MOUNT - Command ID 0x00
142
-----------------------
143
 
144
Format:
145
Standard header followed by:
146 182 winston
Bytes 4+: 16 bit version number, little endian, LSB = minor, MSB = major
147
          NULL terminated string: mount location
148 178 winston
          NULL terminated string: user id (optional - NULL if no user id)
149
          NULL terminated string: password (optional - NULL if no passwd)
150
 
151
Example:
152
 
153
To mount /home/tnfs on the server, with user id "example" and password of
154 182 winston
"password", using version 1.2 of the protocol:
155
0x0000 0x00 0x00 0x02 0x01 /home/tnfs 0x00 example 0x00 password 0x00
156 178 winston
 
157 182 winston
To mount "a:" anonymously, using version 1.2 of the protocol:
158
0x0000 0x00 0x00 0x02 0x01 a: 0x00 0x00 0x00
159 178 winston
 
160 182 winston
The server responds with the standard header. If the operation was successful,
161
the standard header contains the session number, and the TNFS protocol
162
version that the server is using following the header, followed by the
163
minimum retry time in milliseconds as a little-endian 16 bit number.
164
Clients must respect this minimum retry value, especially for a server
165
with a slow underlying file system such as a floppy disc, to avoid swamping
166
the server. A client should also never have more than one request "in flight"
167
at any one time for any operation where order is important, so for example,
168
if reading a file, don't send a new request to read from a given file handle
169
before completing the last request.
170 178 winston
 
171 182 winston
Example: A successful MOUNT command was carried out, with a server that
172
supports version 2.6, and has a minimum retry time of 5 seconds (5000 ms,
173
hex 0x1388). Session ID is 0xBEEF:
174 178 winston
 
175 182 winston
0xBEEF 0x00 0x00 0x00 0x06 0x02 0x88 0x13
176 178 winston
 
177 182 winston
Example: A failed MOUNT command with error 1F for a version 3.5 server:
178
0x0000 0x00 0x00 0x1F 0x05 0x03
179 178 winston
 
180
UMOUNT - Command ID 0x01
181
------------------------
182
 
183
Format:
184
Standard header only, containing the connection ID to terminate, 0x00 as
185
the sequence number, and 0x01 as the command.
186
 
187
Example:
188
To UMOUNT the filesystem mounted with id 0xBEEF:
189
 
190
0xBEEF 0x00 0x01
191
 
192
The server responds with the standard header and a return code as byte 4.
193
The return code is 0x00 for OK. Example:
194
 
195
0xBEEF 0x00 0x01 0x00
196
 
197
On error, byte 4 is set to the error code, for example, for error 0x1F:
198
 
199
0xBEEF 0x00 0x01 0x1F
200
 
201
DIRECTORIES - Opening, Reading and Closing
202
==========================================
203
Don't confuse this with the ability of having a directory heirachy. Even
204
servers (such as a +3 with a floppy) that don't have heirachical filesystems
205
must support cataloguing a disc, and cataloguing a disc requires opening,
206
reading, and closing the catalogue. It's the only way to do it!
207
 
208 182 winston
OPENDIR - Open a directory for reading - Command ID 0x10
209 178 winston
--------------------------------------------------------
210
 
211
Format:
212 185 winston
Standard header followed by a null terminated absolute path.
213
The path delimiter is always a "/". Servers whose underlying
214
file system uses other delimiters, such as Acorn ADFS, should
215
translate. Note that any recent version of Windows understands "/"
216
to be a path delimiter, so a Windows server does not need
217 178 winston
to translate a "/" to a "\".
218 185 winston
Clients should keep track of their own current working directory.
219 178 winston
 
220
Example:
221 182 winston
0xBEEF 0x00 0x10 /home/tnfs 0x00 - Open absolute path "/home/tnfs"
222 178 winston
 
223
The server responds with the standard header, with byte 4 set to the
224
return code which is 0x00 for success, and if successful, byte 5
225
is set to the directory handle.
226
 
227
Example:
228 182 winston
0xBEEF 0x00 0x10 0x00 0x04 - Successful, handle is 0x04
229
0xBEEF 0x00 0x10 0x1F - Failed with code 0x1F
230 178 winston
 
231 182 winston
READDIR - Reads a directory entry - Command ID 0x11
232 178 winston
---------------------------------------------------
233
 
234
Format:
235
Standard header plus directory handle.
236
 
237
Example:
238 182 winston
0xBEEF 0x00 0x11 0x04 - Read an entry with directory handle 0x04
239 178 winston
 
240
The server responds with the standard header, with the sequence number
241
set to the directory entry number. Following the standard header is
242
a NULL terminated string, which is the directory entry. Example:
243 182 winston
0xBEEF 0x00 0x11 . 0x00 - Directory entry for the current working directory
244
0xBEEF 0x00 0x11 .. 0x00 - Directory entry for parent
245
0xBEEF 0x00 0x11 foo 0x00 - File named "foo"
246 178 winston
 
247
If the end of directory is reached, or other error is reached, the
248
header is sent as above, but byte 4 is set to 0x00 and byte 5 is set to
249
the code (either EOF or an error):
250 182 winston
0xBEEF 0x00 0x11 0x00 0x01 - EOF
251
0xBEEF 0x00 0x11 0x00 0x1F - Error code 0x1F
252 178 winston
 
253 182 winston
CLOSEDIR - Close a directory handle - Command ID 0x12
254 178 winston
-----------------------------------------------------
255
 
256
Format:
257
Standard header plus directory handle.
258
 
259 182 winston
Example, closing handle 0x04:
260
0xBEEF 0x00 0x12 0x04
261 178 winston
 
262
The server responds with the standard header, with byte 4 set to the
263
return code which is 0x00 for success, or something else for an error.
264
Example:
265 182 winston
0xBEEF 0x00 0x12 0x00 - Close operation succeeded.
266
0xBEEF 0x00 0x12 0x1F - Close failed with error code 0x1F
267 178 winston
 
268 182 winston
-------------------------------------------------------------------------------
269
 
270 188 winston
FILE OPERATIONS
271
===============
272
These typically follow the low level fcntl syscalls in Unix (and Win32),
273
rather than stdio and carry the same names. Note that the z88dk low level
274
file operations also implement these system calls. Also, some calls,
275
such as CREAT don't have their own packet in tnfs since they can be
276
implemented by something else (for example, CREAT is equivalent
277
to OPEN with the O_CREAT flag). Not all servers will support all flags
278
for OPEN, but at least O_RDONLY
279
 
280
OPEN - Opens a file - Command 0x20
281
----------------------------------
282
Format: Standard header, flags, filemode, then the null terminated filename.
283
Flags are a bit field.
284
The filemodes are:
285
 
286
O_RDONLY        0x01    Open read only
287
O_WRONLY        0x02    Open write only
288
O_RDWR          0x03    Open read/write
289
 
290
The flags are:
291
 
292
O_APPEND        0x01    Append to the file, if it exists (write only)
293
O_CREAT         0x02    Create the file if it doesn't exist (write only)
294
O_EXCL          0x04    With O_CREAT, returns an error if the file exists
295
O_TRUNC         0x08    Truncate the file on open for writing
296
 
297
Examples:
298
Open a file called "/foo/bar/baz.bas" for reading:
299
 
300
0xBEEF 0x00 0x20 0x01 0x00 /foo/bar/baz.bas 0x00
301
 
302
Open a file called "/tmp/foo.dat" for writing, creating the file but
303
returning an error if it exists:
304
 
305
0xBEEF 0x00 0x20 0x02 0x06 /tmp/foo.dat 0x00
306
 
307
The server returns the standard header and a result code in response.
308
If the operation was successful, the byte following the result code
309
is the file descriptor:
310
 
311
0xBEEF 0x00 0x20 0x00 0x04 - Successful file open, file descriptor = 4
312
0xBEEF 0x00 0x20 0x01 - File open failed with "permssion denied"
313
 
314
READ - Reads from a file - Command 0x21
315
---------------------------------------
316
Reads a block of data from a file. Consists of the standard header
317
followed by the file descriptor as returned by OPEN, then a 16 bit
318
little endian integer specifying the size of data that is requested.
319
The server will only reply with as much data as fits in the maximum
320
TNFS datagram size of 1K. If there is less than the size requested
321
remaining in the file, the server will return the remainder of the file.
322
Subsequent READ commands will return the code EOF.
323
 
324
Examples:
325
Read from fd 4, maximum 256 bytes:
326
 
327
0xBEEF 0x00 0x21 0x04 0x00 0x01
328
 
329
The server will reply with the standard header, followed by the single
330
byte return code, the actual amount of bytes read as a 16 bit unsigned
331
little endian value, then the data, for example, 256 bytes:
332
 
333
0xBEEF 0x00 0x21 0x00 0x00 0x01 ...data...
334
 
335
End-of-file reached:
336
 
337
0xBEEF 0x00 0x21 0x21
338
 
339
WRITE - Writes to a file - Command 0x22
340
---------------------------------------
341
Writes a block of data to a file. Consists of the standard header,
342
followed by the file descriptor, followed by a 16 bit little endian
343
value containing the size of the data, followed by the data. The
344
entire message must fit in a single datagram.
345
 
346
Examples:
347
Write to fd 4, 256 bytes of data:
348
 
349
0xBEEF 0x00 0x22 0x04 0x00 0x01 ...data...
350
 
351
The server replies with the standard header, followed by the return
352
code, and the number of bytes actually written. For example:
353
 
354
0xBEEF 0x00 0x22 0x00 0x00 0x01 - Successful write of 256 bytes
355
0xBEEF 0x00 0x22 0x06 - Failed write, error is "bad file descriptor"
356
 
357
CLOSE - Closes a file - Command 0x23
358
------------------------------------
359
Closes an open file. Consists of the standard header, followed by
360
the file descriptor. Example:
361
 
362
0xBEEF 0x00 0x23 0x04 - Close file descriptor 4
363
 
364
The server replies with the standard header followed by the return
365
code:
366
 
367
0xBEEF 0x00 0x23 0x00 - File closed.
368
0xBEEF 0x00 0x23 0x06 - Operation failed with EBADF, "bad file descriptor"
369
 
370 197 winston
 
371
STAT - Get information on a file - Command 0x24
372
-----------------------------------------------
373
Reads the file's information, such as size, datestamp etc. The TNFS
374
stat contains less data than the POSIX stat - information that is unlikely
375
to be of use to 8 bit systems are omitted.
376
The request consists of the standard header, followed by the full path
377
of the file to stat, terminated by a NULL. Example:
378
 
379
0xBEEF 0x00 0x24 /foo/bar/baz.txt 0x00
380
 
381
The server replies with the standard header, followed by the return code.
382
On success, the file information follows this. Stat information is returned
383
in this order. Not all values are used by all servers.
384
 
385
File mode       - 2 bytes: file permissions - little endian byte order
386
uid             - 2 bytes: Numeric UID of owner
387
gid             - 2 bytes: Numeric GID of owner
388
size            - 4 bytes: Unsigned 32 bit little endian size of file in bytes
389
atime           - 4 bytes: Access time in seconds since the epoch, little end.
390
mtime           - 4 bytes: Modification time in seconds since the epoch,
391
                           little endian
392
ctime           - 4 bytes: Time of last status change, as above.
393
uidstring       - 0 or more bytes: Null terminated user id string
394
gidstring       - 0 or more bytes: Null terminated group id string
395
 
396
Fields that don't apply to the server in question should be left as 0x00.
397
The ┬┤mtime' field and 'size' fields are unsigned 32 bit integers.
398
The uidstring and gidstring are helper fields so the client doesn't have
399
to then ask the server for the string representing the uid and gid.
400
 
401
File mode flags will be most useful for code that is showing a directory
402
listing, and for programs that need to find out what kind of file (regular
403
file or directory, etc) a particular file may be. They follow the POSIX
404
convention which is:
405
 
406
Flags           Octal representation
407
S_IFMT          0170000         Bitmask for the file type bitfields
408
S_IFSOCK        0140000         Is a socket
409
S_IFLNK         0120000         Is a symlink
410
S_IFREG         0100000         Is a regular file
411
S_IFBLK         0060000         block device
412
S_IFDIR         0040000         Directory
413
S_IFCHR         0020000         Character device
414
S_IFIFO         0010000         FIFO
415
S_ISUID         0004000         set UID bit
416
S_ISGID         0002000         set group ID bit
417
S_ISVTX         0001000         sticky bit
418
S_IRWXU         00700           Mask for file owner permissions
419
S_IRUSR         00400           owner has read permission
420
S_IWUSR         00200           owner has write permission
421
S_IXUSR         00100           owner has execute permission
422
S_IRGRP         00040           group has read permission
423
S_IWGRP         00020           group has write permission
424
S_IXGRP         00010           group has execute permission
425
S_IROTH         00004           others have read permission
426
S_IWOTH         00002           others have write permission
427
S_IXOTH         00001           others have execute permission
428
 
429
Most of these won't be of much interest to an 8 bit client, but the
430
read/write/execute permissions can be used for a client to determine whether
431
to bother even trying to open a remote file, or to automatically execute
432
certain types of files etc. (Further file metadata such as load and execution
433
addresses are platform specific and should go into a header of the file
434
in question). Note the "trivial" bit in TNFS means that the client is
435
unlikely to do anything special with a FIFO, so writing to a file of that
436
type is likely to have effects on the server, and not the client! It's also
437
worth noting that the server is responsible for enforcing read and write
438
permissions (although the permission bits can help the client work out
439
whether it should bother to send a request).
440
 
441 182 winston
LIST OF VALID RETURN CODES
442
==========================
443
Note not all servers may return all codes. For example, a server on a machine
444
that doesn't have named pipes will never return ESPIPE.
445
 
446
ID      POSIX equiv     Description
447
0x00                    Success
448
0x01    EPERM           Operation not permitted
449
0x02    ENOENT          No such file or directory
450
0x03    EIO             I/O error
451
0x04    ENXIO           No such device or address
452
0x05    E2BIG           Argument list too long
453
0x06    EBADF           Bad file number
454
0x07    EAGAIN          Try again
455
0x08    ENOMEM          Out of memory
456
0x09    EACCES          Permission denied
457
0x0A    EBUSY           Device or resource busy
458
0x0B    EEXIST          File exists
459
0x0C    ENOTDIR         Is not a directory
460
0x0D    EISDIR          Is a directory
461
0x0E    EINVAL          Invalid argument
462
0x0F    ENFILE          File table overflow
463
0x10    EMFILE          Too many open files
464
0x11    EFBIG           File too large
465
0x12    ENOSPC          No space left on device
466
0x13    ESPIPE          Attempt to seek on a FIFO or pipe
467
0x14    EROFS           Read only filesystem
468
0x15    ENAMETOOLONG    Filename too long
469
0x16    ENOSYS          Function not implemented
470
0x17    ENOTEMPTY       Directory not empty
471
0x18    ELOOP           Too many symbolic links encountered
472
0x19    ENODATA         No data available
473
0x1A    ENOSTR          Out of streams resources
474
0x1B    EPROTO          Protocol error
475
0x1C    EBADFD          File descriptor in bad state
476
0x1D    EUSERS          Too many users
477
0x1E    ENOBUFS         No buffer space available
478
0x1F    EALREADY        Operation already in progress
479
0x20    ESTALE          Stale TNFS handle
480 185 winston
0x21    EOF             End of file
481 182 winston
0xFF                    Invalid TNFS handle
482