Current events
Another hardware bug found...
Or at least, certain "bad interactions" with some Z80 processors.
It started last week. I'm beginning to implement snapshot saving, and I just wanted to sanity check things (make sure the NMI button didn't crash games, and that kind of thing). And...it did crash games. It even crashed Manic Miner. Since the NMI button only (is supposed to) result in one new entry pushed onto the stack - which is unavoidable - it shouldn't crash Manic Miner, which doesn't do anything particularly fancy and is not tight on stack space.
First, I investigated the software. No problems found - I wrote a little test to do a "before" and "after" register test, to make sure no registers had been disturbed - none had. I looked at the code, and couldn't see anything that would cause a crash on NMI return. However, I noticed one curious thing - there would be a noticeable delay before the NMI menu came up (on the order of 1/4 second or so) at times when Manic Miner was running. Occasionally, I saw the NMI menu half paint, the screen clear, then the NMI menu repaint. That to me said an obvious double (or more) triggering. The really odd thing was it hardly ever happened when sitting in BASIC, or running a program to try and test the conditions before and after NMI. However, with a game loaded, multiple NMI triggering would happen all the time.
Fortuitously, I had managed to obtain a digital storage oscilloscope. It arrived yesterday. So the first real use of the new scope - see if the NMI signal was "bouncing" and causing multiple NMIs. First, a bit of background to how the Spectranet makes an NMI - basically, when you hit the switch, there's some debounce logic in the CPLD (which is very, very simple - basically a flip flop that goes low until the CPLD decodes a RETN instruction). The output of this flip flop is an open drain pin on the CPLD, which pulls down on the NMI line on the Spectrum. The Z80's NMI is level triggered, and it should not trigger again until the NMI signal has returned high. What I guessed was happening was that the NMI was going low and then "bouncing", instead of staying at 0v, perhaps crossing the logic 1 threshold of the Z80 numerous times before settling. However, within seconds of connecting the new 'scope, that was proven to be unequivocally untrue - the signal was as perfect as can be. (See the picture to the top right). I decided to have a look at the DivIDE, and see what its NMI signal looked like, since I hadn't heard any complaints about crashes being caused by the DivIDE's NMI button. Sure enough, it gets returned back to a logic high level. I suspect the M1 cycle subsequent to the NMI being triggered probably resets the DivIDE's NMI output (it resets between 15 and 23 T-states after being asserted). You can see the NMI output of the DivIDE in the picture on the bottom right.
The upshot of this is I've got to squeeze more logic into the already pretty full CPLD to reset the NMI output but do it in a way that it can't be reinvoked until the RETN instruction has been seen on the bus. And again, the reason I had never seen this problem is because it's not a problem on the Spectrum +3. I can invoke the Spectranet's NMI as often as I like while playing Manic Miner, and it never crashes - so obviously, different Z80s behave a bit differently to an NMI signal that's held low for a long time.
Winston 21:30, 30 October 2009 (GMT)
Snapshots
A feature that I found would be important at RetroReunited (on talking to a few other people) is the ability to snapshot - to be able to load and save snapshots to a filesystem.
To start, I've done the easy one - load from a 48K .SNA file. Surprisingly, this worked first time. There's a BASIC command to do it now, named %loadsnap. (Ultimately, it'll detect the kind of snapshot, and do the right thing). For loading, I intend to support at least 48K and 128K .SNA files, and probably .Z80 files too. For saving, I'll just support .SNA because it's the most straightforward to support, and should do the job fine for nearly all uses.
This also requires an overhaul of the NMI menu. Snapshotting will be done from the NMI menu as you'd expect, but it's going to be done from a module. Currently the NMI menu is hard coded in the utility ROM. What it needs to do is to go through all the ROM modules, to see if they have an NMI action (in which case, print an string and allow a jump to the address of whatever routine will handle the NMI).
Incidentally, as for the next "Spectranet World Tour" event, I'm already planning for the Vintage Computer Fair, which will be held at the historic Bletchley Park next June. I intend to have a LAN of at least 4 Spectrums set up and running, plus the VAX fileserver I had at RetroReunited.
Winston 18:55, 25 October 2009 (GMT)
Making streams usable
After much work here and there, BASIC streams support now is a bit more worthwhile. Not only does it support TCP sockets, but now also reading/writing files on a file system, as well as reading directories. A brief summing up of what's been done to make streams into something practically useful:
- Bug fixes. The TNFS module wasn't freeing up file/directory descriptors on close (whoops!)
- Better memory management in the streams module itself (of the memory allocated by BASIC's MAKE-ROOM routine)
- File and directory support
- Lots and lots of testing!
Supporting files and directories, is I think quite a big thing. It means the BASIC programmer can take advantage of any filesystem module that's written - for instance, write a HTTP filesystem, and BASIC can use it. Write an FTP filesystem, and BASIC can use it. Write an IDE filesystem for an attached DivIDE, and BASIC can use it - and no programs have to be modified. This of course was the original intention of streams in the BASIC ROM, but the support was never finished back in the day. I hope the Spectranet can show the true beauty of what was actually intended back in the day.
There are other things I need to support, and I want to try and support without a proliferation of BASIC extensions, by means of streams. I still need to write a control stream which can be opened to find status of other streams. The syntax of INPUT# is actually quite handy for this kind of thing. For instance, imagine we have a stream that can stat (find information) on a file. A line in BASIC could be written as such to use a control stream to do this. Imagine the "stat" command returns the size in bytes, followed by the type of file, followed by the permissions...
INPUT #4, "stat /path/to/some/file";size;t$;p$
(It's a pity that BASIC only allows single characters for string variable names).
The other thing that I've made streams support is EOF (end of file) handling not by returning an error to BASIC (which makes your program bale out with the unhelpful error '8 End of file', at which point your program has stopped and left all the channels open...) The usual idiom in a language like C for reading a file or something you fully and correctly expect to stop reading at EOF looks something like this (in C-like pseudocode):
filehandle=fopen("/path/to/file", "r");
while(data = read(filehandle))
{
do_stuff();
}
fclose(filehandle);
And handily, when we hit EOF (or some other error condition) the while loop exits, and we get to close the file handle and do whatever else we need to do. Normally, Spectrum BASIC does not allow this - you just end up fallling over with 8 End of file. However, Spectranet streams support now allows you to at least do this - using the %oneof (on end of file) command:
10 %fopen #4,"filename","r" 20 %oneof 100 30 INPUT #4;a$ 40 ...do something ... 50 GO TO 30 100 %close #4
As a demonstrator and aid to testing it works as I meant it to, I wrote a small games menu program. It reads a list of games and their filenames from a file, then allows you to press a key to launch whichever game you choose (the game, of course, is in a .TAP file). This also brought up a few other issues...
The most significant is that when you %fopen (or %connect or %opendir or whatever) a stream is that some memory must be allocated to store the stub that makes a call into the Spectranet streams module to actually do the work. Memory management in the ZX ROM is fairly basic, to say the least - you can call MAKE-ROOM to make some space somewhere (for streams, just below the BASIC program itself in memory), and RECLAIM-1 to free that memory. However, I can't just call RECLAIM-1 whenever a stream gets closed without a great deal of shuffling stuff around - and the other complication is that if someone else has done a MAKE-ROOM, my code won't know about it. This means calling RECLAIM-1 automatically when a stream is closed something that may tread on someone else's code. So at the moment, the Spectranet streams module just keeps a note of what memory it has allocated, and tries to reuse it (for instance, if you open a stream and close it, RECLAIM-1 doesn't get called, but the streams module knows it allocated space for the stub, so the next time you open a stream, it just reuses this already allocated space for the new stub). Instead, if the programmer wants this room reclaimed by the ZX ROM, they explictly need to tell the streams module to do it. The programmer, after all, is the only person who will know if they did other non-Spectranet streams stuff there. Most of the time, the room will never need to be permanently reclaimed. But there are instances where it's essential.
In the case of my games loader, if you don't actually reclaim the room - many games just won't load - they die with "M RAMTOP no good" when they try CLEAR xxxxx. So, when a game is selected in my BASIC game loader menu, the command:
%reclaim
is executed, which makes the streams module deallocate all memory from the original value of PROG to whatever the latest value of PROG was when it made the last call to MAKE-ROOM.
What's next? Probably a poll stream (in which you can do INKEY$# without actually reading off a character, which will tell the programmer which stream has stuff ready to read), and a control stream to allow other more complex things to be done with INPUT#. I also intend to make a video this weekend, showing what's been done so far.
Winston 20:55, 20 October 2009 (BST)
Improving tape emulation
A while back I added tape traps to the Spectranet, meaning you can put a TAP file on a network filesystem, and load it. It's similar functionality to what the DivIDE has, and what is available in ResiDOS, except you're loading a TAP file over the network rather than directly off an IDE device.
I decided "wouldn't it be nice if there were a default file", so if you just typed LOAD "", it would load this file. So I added this - typing LOAD "" will try to load a file called "boot" in the current directory. The file "boot" is a TAP file (just because it doesn't end in .TAP doesn't mean it's not a TAP file!) - because the format of files saved using the %save command is TAP - it's how the metadata associated with a file is stored. (I made the standard file format be TAP because then it makes it easy to transfer files from a Spectrum to an emulator and vice versa. And TAP holds all the information that's needed).
So what you can do is this. You can write something like a menu in BASIC, and %save it as "boot". Then when you type LOAD "", it'll load this BASIC program. Or, on a 128K machine you can use the Loader. (This unfortunately doesn't work on a +3 or +2A owing to the different way they arranged the ROMs).
The idea of this is what I learned by watching people at RetroReunited - if they reset the Spectrum, they'd always do something like bring up the loader or type LOAD "" expecting something to happen. So I decided something ought to happen if you did do that!
Then there were a couple of bugs - well, not bugs so much, but problems with certain games. For example, Starstrike and Skool Daze do something different to many games - and if you've loaded them off tape you'll know that after the SCREEN$ image loads, they don't then start loading another block - the loader simply continues until the game is loaded. What happens, though, is the stack gets overwritten by the tape loader. With a real tape, this is harmless - the right return address got written by the tape file, so when the ROM loader returns, it goes to the right place. (Perhaps an address set up in the tape data itself). But when the tape trap fires, it's a programmable trap and enters via the NMI - and a whole lot of registers get pushed onto the stack. Additionally, the Spectranet replacement for the main rom's LD-BYTES routine (which would normally be loading the bytes off tape) needs to use the stack. Plus the original value of the Spectranet page B register is on the stack. Generally, this means bad things happen when the stack gets overwritten by the TAP file that's being loaded.
The solution: make the NMI routine's stack live in Spectranet RAM. This way, it can't be overwritten this way. Now both Starstrike and Skool Daze load just fine (as well as probably a heap of other games).
However, I did run into an interesting problem I should have thought of - especially as I actually documented it on the wiki when I first added programmable traps! The layout of the ROM got shuffled around a little bit, and this meant the default trap for LOAD "" actually caused a trigger in the Spectranet ROM, because code now happened to execute at the same address as LD-BYTES in the main ROM. So I added a little extra logic to the CPLD to disable traps while the Spectranet ROM is actually paged. As a workaround for users of the other prototypes, I can probably find the right combination of NOPs to make the ROM not hit this address.
Winston 21:19, 29 September 2009 (BST)
The Analogue World
For a great deal of time, I've been doing everything with the Spectranet on my +3. There's a couple of good reasons for this: first, it has RGB out (and all my UHF only Speccies have serious modulator drift problems), and secondly, the floppy disc drive makes it easier to reload the Spectranet ROM if I bork it. (Otherwise, it must be loaded through the tape port while I hold the DISABLETRAP connection at 5v by hand...)
But I got given a Spectrum+ last weekend, and it turns out it has a good modulator (it still drifts, they all do) but it gives good colour output on my TV. But I just got the "pixel vomit" screen when I tried to use it with the Spectranet. I had the same problem with an issue 4 board with the composite mod earlier, but I had put it down to a dirty edge connector. So I tried it on more machines, pixel vomit on all of them except my Issue 2 Spectrum. Which, once warmed up for a few minutes...also gave pixel vomit.
There was obviously some problem with initialization. First, I checked the Spectrum had actually come out of the RESET state by checking the voltage of the RESET line, it was indeed 5 volts. (The Spectrum's RESET circuit is very high impedance, and the CPLD will prevent it from working at all, so it's buffered before reaching the CPLD). I checked various other things, checked that the clock signal was present (recently, I made changes to the clock circuits in the CPLD), I checked that the voltage was adequate, and everything obvious. Then I started up Xilinx ISE and started going through the CPLD design.
One thing I had in mind was the slowness of the reset circuit on these machines - it's just a simple R/C circuit, and I had thoughts that perhaps it was possibly resulting in a bit of "jitter" in the CPLD as the voltage passed through the threshold oh so slowly. I tried all sorts of things to eliminate them - they'd always work the first five or six times (solved it! I thought each time...), but then would inexplicably fail. (As the Spectrum warms up, the R/C reset circuit gets slower). The one place I didn't think to look after banging my head on the table for two days was the memory paging registers in the CPLD. I had forgotten they get reset too (they don't actually need to be reset), and didn't even look at that part of the circuit - until out of complete frustration I had deleted the entire RESET net in the CPLD...
What is happening is the RESET level reaches the Z80's logic high threshold long before it reaches the logic high threshold for the 74HCT1G125 buffer, which feeds the CPLD's reset input. So the CPLD is being held in reset for quite a long time after the Z80 starts running - at least 100,000 T-states! The best fix is to just get rid of the reset circuit from the memory paging registers, but with a few Spectranets already out there with people who don't have the kit to program the CPLD, well, a huge delay loop in software was the pragmatic approach to take. And of course, now it works perfectly on all my Spectrums.
Winston 20:20, 26 September 2009 (BST)
At long last, addressing +3 BASIC
The Spectrum +3 brought some good enhancements to the world of the Speccy (even if a bit flawed - oh for a 3.5in disc instead of the unusual Amstrad 3in unit), but also some irksome compatibility issues. Not only is it incompatible with any peripheral with a ROM which wasn't +3 aware (i.e. anything designed before the +3) due to the change in the edge connector, but the other problem is the way the BASIC interpreter in 128K mode breaks with what happened with the toastrack 128 and the grey +2. In those machines, all the BASIC interpretation was still strictly done by the original interpreter. But in the +3, some work is done in a different ROM - including the RST 8 error handler.
So when you trap an error to extend BASIC, it suddenly doesn't work even though you've made your device electrically compatible with the +3.
I want the Spectranet to smoothly support all of the original machines - that is the rubber 48K machine up to the +3, and anything with a compatible edge connector and compatible ROMs. I want it to smoothly support +3 BASIC as well as 128K BASIC without forcing USR 0 mode.
This is how I'm going to do it.
On power up, the Spectranet will find out whether it's on a machine with the +3 ROMs. This can be done by looking at the first 6 bytes of ROM 1, which contain the string "Syntax" on a +3. It will store this in a Spectranet system variable (i.e. in the 0x3F00 system variables area in the Spectranet's own static RAM).
When the error trap fires, it'll see if this flag is set, and if so, will check to see whether we're in USR 0 mode or in 128K BASIC mode. This can be done by checking for the presence of the paging routine, which is kept where the printer buffer used to be in RAM. USR 0 mode and 48K mode wipes this routine out - so we can use this to tell that we're in 128K BASIC. If we are in 128K BASIC, ROM 3 (the original syntax ROM, and where the ROM routines that the Spectranet BASIC extensions use) will be paged for the duration of the BASIC extension, then restored when we exit, either due to successful completion or some sort of error.
I suspect I'll hit some bumps in the road while doing this (I hit a few while writing the detection routine yesterday). But the upshot of this is that BASIC support for all the machines should "just work".
Also, this week, I've implemented the "default file" concept. If you type LOAD "", it will load the first BASIC block it sees in a file named "boot" in the current working directory (TAP format, as the Spectranet's BASIC filesystem support uses TAP as its container format). This even works with the 128K Loader. What it's doing is trapping the tape loading routine and opening the file "boot" if it's present, and then setting up normal tape traps to load this TAP file. The upshot of this is that the next retro party I go to, it'll be really easy for people to just load stuff over the network - just LOAD "", or on a 128K machine, select the Loader, and away you go. (I'll write a little menu program in BASIC for navigating to the various programs).
Winston 19:55, 17 September 2009 (BST)
The Spectranet world tour continues!
First Oxford, then Bilbao, now Huddersfield!
The Spectranet was on show (with all the latest code enhancements) at Retro Reunited, on the 12th/13th September. The Speccy was paired up with a MicroVAX as a fileserver. (The VAX was also the DNS and DHCP server, unfortunately the hotel had no "wider LAN" with these things, nor an internet connection other than WiFi. I could have set up my old PowerBook to be a router on WiFi, but owing to a shortage of TVs, I only had one Speccy on the network so decided to show off the network filesystem rather than the IRC client!)
Which brings me on to more recent developments.
Since the last update, I've added more code to the BASIC streams support, so you can now write a server program in ZX BASIC. Also, I've made the mount command for mounting filesystems MUCH better - instead of passing half a dozen parameters, which was rather clumsy, you now supply just two - the mount point, and an URL for the filesystem you want to mount, in the usual format - proto://user:passwd@host/some/path . The default protocol at the moment is TNFS, so if you just specify an IP address or hostname, it does an anonymous TNFS mount of the root on that host.
Also, I've now added the ability to mount more than one filesystem at a time - up to four filesystems can be mounted simultaneously. If the filesystems are TNFS, then only one network socket is used for the lot, since TNFS is a very low resource UDP protocol.
What was clear from Retro Reunited is that one of the priority developments for the BASIC module is to add a "default" file to load (this will of course work with the Loader on 128K machines). Of course, people just casually playing stuff - for example, at a show like this, may just want to start the Speccy and have it load a menu of games. The other big feature I think people would like is the ability to save snapshots to a filesystem - a bit like the Multiface - hit the NMI button, then save a snapshot to any of the mounted filesystems.
Winston 22:11, 14 September 2009 (BST)
RetroEuskal, and BASIC streams
No updates for about a month, but I've been a bit busy!
First, I had the grand challenge of demonstrating the Spectranet at RetroEuskal - held inside Euskal Encounter (the second largest LAN party in Spain, with 4096 network ports - most in use). Fortunately, the DHCP client worked fine with the Euskal Encounter DHCP servers, and braved a very busy network segment with vast quantities of broadcast traffic (mostly DHCP requests and ARP who-has requests). The demonstration went well, and people even seemed to understand my undoubtedly heavily accented Spanish...
Secondly, one of the things I'd been working hard on so that I could demonstrate it at RetroEuskal was BASIC streams. I got the prototype code done just in time, and tested it with things like an SMTP client written in ZX BASIC (yes! It worked!). So far, you can write TCP client programs in BASIC. Writing servers is going to take a bit more effort (it'll need a separate control channel so that you can peek at the state of things without reading).
It's now possible to write a program like this in BASIC:
10 %connect #4, "some.computer.com", 2000 20 INPUT #4;"Tell me> ";a$ 30 PRINT "Remote computer said: ";a$ 40 PRINT #4; "You said: ";a$ 50 IF a$="q" THEN GO TO 100 60 GO TO 20 100 PRINT #4; "Bye!" 110 %close #4
Once accessability from BASIC and the network filesystem is done, basically, the base software is actually finished. The end is in sight!
On the hardware side, I'm thinking of getting some PCBs made, with the aim of making sure my modifications for "production" are good. There's one bug that had to be fixed (which won't affect normal users), plus I want to try and make a small mod to the routing of the ethernet traces from the magjack to try and get rid of a couple of vias, sometimes the W5100 has trouble getting a stable connection on reset, and I want to eliminate that from my enquiries. Also I want to put another jumper in so the user can select the voltage source for the 3.3v regulator - either the 5V from the Spectrum (works on all models), or the 9V (48K only) - the reasoning, to lessen the load on a rubber key Speccy's 7805 regulator.
Winston 21:33, 6 August 2009 (BST)
A whole lot of new excitement
Just over a week ago, you may have seen the news that the Speccy made its first tweet on Twitter. The client was hacked together in the Gloucester Arms in Oxford, over a pint of Black Beauty (you do get some odd looks with a Speccy on the pub table, but it was the CSS meet!)
But other than that, more exciting times: I've put together a basic automounter (it needs much more work to do what I eventually want it to do), so that filesystems may be mounted at boot/reset time. So the Spectrum can now be powered on, and you can immedately start using the network filesystem. Also, a comprehensive "%info" command has been added to the Spectranet BASIC ROM module which gives information on files. For generic non-Spectrum files (and directories) it just shows the basics, such as filesize (and I intend to show file mode too). But for Spectrum files (the native format being used is TAP), it lists the contents - so, if you have a complete TAP file for a game for instance, it'll show all the blocks that are contained in the TAP file. Also, for testing, I wrote a program in BASIC to give a game loader menu, and then load the game from a TAP file. I wrote it on a real Spectrum, and trusted the TNFS code enough to save it over the network!
There are some games that don't seem to load properly - I need to investigate whether this is just a +3 incompatibility, or whether there are bugs in the tape traps.
Other than that, I've fixed a couple more bugs that I discovered in the base ROM.
Next I want to add support for BASIC streams. (Andrew Owen will like that of course!) For the time being, I intend to add a few commands using the RST 8 trap for opening a socket - it has been suggested that a control channel can be used for sending commands, but that involves a lot of extra work and I'd like to have at least TCP client streams working to demo at RetroEuskal (now wouldn't it be nice to do a Twitter client in ZX BASIC!)
Winston 21:58, 7 July 2009 (BST)