Old News (June 08 - July 08)
BASIC extended
I've put the infrastructure in place for BASIC extensions in the ROM this weekend.
By default, BASIC is not extended. The code in the fixed ROM page (page 0, mapped to 0x0000-0x0FFF) contains the functions required to handle RST 8 traps, interpret new commands, and dispatch them to the routine that carries out the actions - but unless some other ROM module (or code in RAM for that matter) adds a command to interpret, it simply returns control back to the Spectrum's RST 8 routine. To make it easy to extend BASIC, all a programmer needs to do is pass a 6 byte structure to the ADDBASICEXT routine. The structure contains the error code the extension responds to, a pointer to the string that makes up the command keyword, the ROM page (if required), and the address to call to handle the new command. When RST 8 is trapped, the code in the main ROM goes through this table to see if a new command is to be invoked. If it finds one, it pages the required memory page (if specified) and then calls the address in the table.
The purpose of the extensions is to allow for things such as a network file system. Not everyone will have mass storage, and the Spectranet is expected to do something useful as a stand alone expansion. If you look at the Spectranet article here, you'll see one of my goals is to provide a simple network filesystem.
At this point it's worth introducing how the Spectranet is expanded. There's 128K of flash ROM, of which only 16K is reserved for system usage - the rest can be used by other developers to provide ROM modules. Using the ADDBASICEXT routine, these can now also provide extra commands. Each 4K page of ROM can have a vector table associated with it, allowing things such as power-on/reset time initialization etc. The core Spectranet modules will also use this mechanism - although they might live in the reserved area of the flash memory, they don't behave much differently.
The fixed 4K ROM area is now almost filled - with the BASIC extension infrastructure in place there's only a couple of hundred bytes left for code...so I'll have to use these wisely.
The next bit to look at is channels and streams so that networked programs can be written in BASIC. A channel in the context of the BASIC ROM is a device that's expected to always be there, so channels don't really match what sockets do, so there will have to be a little trickery involved. My ideas so far are to create a channel on boot and open a stream to it right then - Andrew Owen suggested channel 15 would be best for this. This would form the control channel. BASIC programs would be able to open and close sockets by sending data to stream #15, which would be a pipe to a small parser that would interpret the data that was sent. The Spectranet ROM code would then handle creating a channel that connected to a socket, then opening a stream attached to this channel. This way, RST 8 traps wouldn't be needed to handle channels/streams at all - all that would be needed is a lightweight wrapper in RAM that dispatches the calls to the right bit of ROM via the CALL 0x3FF8+ entry points. Avoiding RST 8 traps for basic functionality such as this ensures that something useful can be done in case another board traps RST 8 - only one peripheral can be an RST 8 trapper at any one time!
Winston 23:23, 20 July 2008 (BST)
Starting on the last bits of the ROM
There's one last piece of base ROM infrastructure that I think needs to be in place before I hand out prototypes as 'developer boards' - the RST 8 trap.
I had a go at this thorny issue quite a while ago, and got frustrated with it. I'm not familiar enough with the Spectrum ROM is the main stumbling block. Things are a bit easier now, though - Garry Lancaster gave me some information that makes things a bit clearer, and I've re-read the IF1 ROM disassembly a few more timss.
What I've done is to put a bit of code into the RST 8 routine to call code anywhere in memory, so I can load experimental code into RAM (over the network, naturally!) rather than flashing the ROM each time I want to try something new. So far, basically, my experimental code (which runs from RAM) is mostly what the IF1 ROM does. Error handling now works - after a minor struggle, but by looking at Garry's notes and fiddling with the code, eventually it seemed to work OK. I have two problems though: for a while, I had successfully parsed commands working OK - but now they've stopped (it hangs on return). Also, successful parsing crashes +3 BASIC hard, even though error handling works - so exiting back to the Spectrum ROM obviously needs to be handled a bit differently on the +3. I suspect there's a problem with which ROMs are paged in. I've been unable to test on a 128K+ (toast rack) because mine's broken at the moment and I've not found a schematic for the machine (the 128K service manual at WOS only has schematics for 48K machines!)
What I ultimately want to do is draw up a fairly detailed spec on how RST 8 should be handled, and then write it from scratch to avoid infringing Amstrad's copyright on the IF1 ROM. This requires me to get a pretty thorough understanding of what is actually going on.
Also, discussions on the WOS forum about the need for RST 8 trapping at all have occurred. To summarize: Andrew Owen says that all basic network functions (i.e sockets) can be done using channels and streams - so why have RST 8 traps at all? Andrew has made a convincing argument that the socket library can be implemented in BASIC this way. However, I think RST 8 traps are still needed for "application" functionality - for example, extra commands for a network filesystem, or to launch an FTP client etc. So I think the infrastructure should be there and working in the core ROM. I strongly agree with Andrew on using channels and streams for the socket library - it's simply the proper way to do it and it will avoid complications with other peripherals if accessing the socket library from BASIC does not need to use RST 8. I will have to see if he wants to write that code now. Perhaps I can bribe him with a free prototype board :-)
Aside from that, I am going to see if I can fit in a software enable/disable bit for RST 8 trapping into the CPLD - I meant to have one all along, but it got forgotten somewhere along the way. It might not fit though!
Winston 20:31, 17 July 2008 (BST)
The last tweaks
After the previous session of butting heads with the CPLD, I decided to see what logic could be cleaned up. This resulted in the removal of the 0x0000 trap - since it's conditional on reset being pressed (or the power up reset), it's not needed - the trap can be done just on reset instead. This freed up enough room in the CPLD for complete 16 bit I/O port decoding.
I did think I was going to have to sacrifice the RETN decode for this because things are so tight, but fortunately this turned out not to be the case (the RETN decoder could be replaced by a bit of logic to unlatch the NMI condition latch off the status/control register - and requiring the programmer to read the status/control register, reset the NMI condition latch bit and then write it out again, but decoding RETN is a much more elegant way to do this).
Incidentally, the programmable trap is now documented here: Trapping execution
Winston 10:04, 7 July 2008 (BST)
Final hardware features, and some documentation
Updates haven't been so thick and fast the last couple of weeks, because I've taken a little time off the project. I find if I use all my spare time on a project ad infinitum, I end up getting burned out on it, so I need to take a break every so often and do other stuff.
However, this is good time for reflection. One piece of reflection was 'what to do for the last execution trap'. To recap, one function of the CPLD is to intercept CPU instruction fetches at certain addresses, and page in the Spectranet memory. It does this, for example, if RESET is detected (the reset button or power up), and the CPU executing code at memory location 0x0000. This is so the Spectranet can do its initialization. Similarly, there's a trap for 0x0066 (NMI), 0x0008 (RST 8 - so new BASIC commands can be added). There's also a trap at 0x007C which does the reverse - unpages the Spectranet memory, so the Spectrum's current main ROM gets paged back in. Having some time to reflect about the last trap address to implement - I decided, wouldn't it be a lot better if it wasn't fixed, and the programmer could set it? After all, simplifying the memory pager freed up an I/O address that could be used to program the trap address...
After some thought, I decided the best way to implement it was instead of controlling paging, this trap instead fires an NMI. This means you can trap any address - execution in ROM or RAM - it doesn't matter! This is because firing an NMI won't depend on the program counter being in the lower 16K, the only place Spectranet memory can be paged. By firing an NMI, even if the trap address is in RAM, it doesn't matter - because the CPU will set PC to 0x0066, and the fixed NMI trap can take care of the paging. This gives the greatest degree of flexibility.
So how do you tell if the NMI was caused by someone pressing the button, or by an execution trap? Simple - examine the return address, which will be the first thing on the stack. If it's set to the next instruction after your trap, it was your programmable trap that caused the NMI. Anything else, and someone pressed the NMI button. Ah! But there's a finite chance that someone could press the button just at the moment the trap would occur, you may cry. Well, in that case - who cares - the NMI would have fired anyway, and it effectively just means that code traps have a higher priority than the NMI button.
Implementing this did cause a bit of a headache, though - I ran out of CPLD resources. Strangely, not macrocells, product terms or function block inputs - but it got into a state where essentially (as I understand it) the software can't route the logic network. So I told the fitter to make exhaustive attempts at fitting it into the device... which resulted me playing a lot of 'Elite' on the BBC Micro while waiting for it to churn. It still didn't fit.
Sacrifices therefore had to be made.
I had another trap set up for 0x0038, the maskable interrupt, together with some conditional logic. This trap would only be entered if the W5100 had asserted the INT line (which isn't directly connected to the Spectrum's Z80 - it goes through the CPLD first so to be gated) and the programmer had set the INT_ENABLE flip flop in the CPLD. However, the maskable interrupt trap has turned out to be rather less needed than I originally thought, when I started on the project - it's not been used. The trap was provided as an easy way to tell the difference between a W5100 interrupt and the 50Hz ULA interrupt, however, if someone wants to do interrupt driven things, they can do an IM2 routine and check the state of the W5100's interrupt register. The 0x0038 trap wouldn't save much - and it certainly wouldn't be anywhere near as useful as a programmable trap that could be set for any memory address.
Even with the 0x0038 trap and its associated I/O pins gone, the fitter still can't fit the design into the CPLD using the default settings - it has to have a few tries before it comes up with a configuration that fits. Unfortunately, there is one CPLD function that's still needed - complete 16 bit port decoding...hopefully that can be shoehorned in (currently, only the lower half of the address bus is decoded for I/O). So no more new hardware features, without using an XC95144. (I could make it really fancy with an 95144).
The new programmable trap is programmed very easily:
- Send the LSB of the address to trap to port 0x80ED (note this port number is going to change)
- Send the MSB of the address to trap to port 0x80ED (the same port)
- Enable programmable traps by setting bit 4 of the control register (0x80EF)
Other than this, if you've been watching 'Recent Changes' (note you can get an RSS feed for this) you may have seen some tutorial pages coming together. Before I can start handing out prototypes to people who want to develop, I need enough documentation available that people can easily get up to speed with doing things with the hardware. So far, I've done an overview tutorial, plus two practical tutorials - how to make a simple TCP based server and client.
I also built the fifth board, which went pretty smoothly - I got much better at using the right amount of solder paste.
Winston 19:25, 5 July 2008 (BST)
Number 4 - The Problem Child
I just assembled the fourth prototype board, and it's been The Problem Child. It was the first assembled 'all in one go', and it didn't really go very well.
First, the flash got misaligned on its pads (so the whole chip was slightly lower than it should have been on the board, and missed its pads, so pin 1 was on pad 2 and so on, with pin 16 sitting on solder mask). So that had to be hot air'd off, cleaned up, and resoldered. I think this was probably the root of many problems - that area of the board went through three reheats and I think it unsoldered a couple of pins on the CPLD (they were on the flash side of the board), and also the flash still didn't solder well - there was one pin that was not making contact. Testing all the pins to find the bad contact is slow and laborious - I spent longer hunting for bad solder joints than I did assembling it in the first place.
Then there was a bad joint on the RD pin of the W5100, although I didn't think that would be the case (I tested pretty much everything before trying the RD pin, because a software readback showed what was written - correctly!) Looking at it closely with a magnifying glass it's also very slightly misaligned, which didn't cause any problems till I resoldered the offending pin, which then formed a short with its neighbour (the RESET pin), and I couldn't clear the short until I stuck the point of a needle between the pins to force them apart.
It took about an hour to put the paste on, and set the components in place, then put hot air on it to solder. It took four hours to actually get it working.
But it works now. Oh for a pick and place machine.
Winston 22:05, 21 June 2008 (BST)
Paging Mr. Memory
In rearranging some of the ROM code (mainly, moving data out from flash page 0 and putting it in page 1 instead, to free up more space for code), it was also time to complete some memory management routines - a method to do calls to other pages ('sideways calls' as such), and also do things like push the last page on the stack, so it can be restored at the end of a routine. For example, the 42-col print routine needs to restore the page that was in page A after it gets done using the character set data.
As I alluded to last time, I also wanted to make some hardware changes to the paging mechanism, which was a bit clumsy (two 8 bit page registers, one for each paging area, and a chip select register, giving a potential memory of 4MB - which is overkill. I initially did it that way on the breadboard because I'd not decided what memory I was going to put on the eventual PCB back when I started prototyping the project). However, the memory is settled, and having two registers that need to be set (one shared between the two paging areas) was clumsy, and a bit wasteful of CPU cycles. I had planned to rationalize it a while ago to just have a single 8 bit paging register for each area, using the upper two bits as the chip select, giving a maximum of 1MB of address space. Since the W5100 has 32K of buffer memory, the static ram is 128K and the flash is also 128K, this is more than adequate.
Since it was going to be extremely clumsy writing the pushpage/poppage/sideways call routines with the old paging scheme, I thought that there was no time like the present to migrate to the final scheme. It went pretty painlessly - I changed all the areas of the ROM code where paging was done to use the new scheme (basically, selecting the page by passing it as a single 8 bit value in A, instead of in register pair HL) - flashed the ROM (over ethernet naturally), reconfigured the CPLD with the new logic, then reset. The hardware worked, but I'd failed to change the ROM initialization code properly in one place so it failed to run the DHCP client on startup. This was an easy fix, and with the aid of a new version of the standalone flash-over-ethernet program (sent to the +3 via the tape port, and saved to floppy), I flashed a revised ROM - and hey, everything worked.
In other hardware news, I sent a Farnell order off at the start of the week and restocked a few components which I was running short of (such as 0.1uF capacitors which I use a lot of - and the 18pF capacitors I ran out of) so this weekend I'll probably assemble a couple more boards. After that I'll get the ROM into a state that I feel that people can at least develop with, and cast around for interested parties who would like to play with the board.
Winston 20:57, 18 June 2008 (BST)
The Third Spectranet
The third Spectranet board is up and running.
The object of this build: build more of the board at once, and see if hot air soldering of the tiny 0603 sized parts was practical.
So armed with a syringe of Edsyn CR44 solder paste and my trusty B&Q hot air gun, I went to work. This board was to be assembled in two stages rather than three - the first stage, solder the CPLD, flash, RAM, reset buffer, ROMCS transistors and all the associated passives (resistors and capacitors) in one go - then program the CPLD and test the memory, then add the W5100, its 74HCT245 buffer, and all the discrete components associated with that end of the board.
I was gratified that the 0603 parts didn't blow away - the solder paste seemed to have enough stiction to keep them in place. Interestingly, when the solder paste began to turn into solder, the components would all automatically centre up on their pads (except for one, which 'tombstoned' - i.e. it sat up on its end, which I corrected by pushing it over with tweezers while still having the hot air on it). One or two components had far too much solder paste which needed mopping up with braid, and of course the inevitable odd bridge on the fine pitch SMD stuff. Visual inspection after mopping up any excess showed the joints all looking good, so I added the edge connector and plugged in, programmed the CPLD and tested the memory - everything checked out OK.
Next was the W5100 and buffer and all the discrete components surrounding those. They all went on trouble free, except some of the capacitors which are lined up (the three bypass capacitors near the 3.3v analogue pin on the W5100, and the pairs of 0.1uF capacitors on the 3.3v digital pins) tended to slide towards each other and stick together. This is only a cosmetic issue since they are all supposed to be in parallel, so I didn't worry too much about it - the main thing is there's a good solder joint. (The way to stop that happening would be to space the components out a little more so there's solder mask between them, or try to use less solder paste).
The next board will be made all at once, since I'm getting the process down. I think the board I just did was about 2 hours work, and I"d like to get this down a bit (probably the best way will be to have a solder paste stencil made up, since putting all those dobs of paste on each pad takes some time). Unfortunately, it won't be made right away because in an ordering blunder, I forgot to restock with 18pF capacitors and I discovered I had used the last two making this board.
So it's back to software next week - I still have quite a few things I need to do to the core ROM code before development versions of the board can be made available, and I also want to change the memory paging scheme... but with continuing hardware successes, it won't be long now!
Winston 23:25, 14 June 2008 (BST)
Making contact with real people
Another small milestone: my Spectrum +3 has made first contact with other live people over the internet. As alluded to last time, one of the example programs that will be available for people to use and developers to tinker with is an IRC client. So, ZX-IRC was born. It's only minimally functional at present - you can join channels, talk to people, quit etc. (and to amuse the administrators of SynIRC, who automatically do a CTCP VERSION request when you connect, it responds to a CTCP VERSION). I don't want to get too bogged down with it, so I'll just add a few more minor features (handle some more messages that ought to be handled, give it a proper keyboard buffer because if a message comes in when you're typing it can drop a keystroke etc.) Unfortunately, the version of z88dk that I have doesn't have an snprintf() function - but I did import OpenBSD's strlcat/strlcpy functions for safer handling of arbitrary strings arriving from the IRC server. Unfortunately, no snprintf means some of the code is a bit cluttered with lists of strlcpy/strlcat calls to build strings - but better that than having some unexpectedly long string overwriting the IM2 vector table!
The next job is to make more hardware.
Winston 21:48, 12 June 2008 (BST)
The Second Spectranet
A key part arrived last week - a reel of solder wick (amongst a bag of other stuff from Farnell). This is a vital part of home surface mount assembly.
I got busy completing the second Spectranet. Again, I did this fairly cautiously - a bit at a time - first, just having a board with the CPLD and 3.3v power circuits, configuring the CPLD, then putting the memory on, and testing the memory, and then the W5100 and its associated parts. This time, I used hot air and solder paste for all the ICs including the static RAM and the buffer for the W5100 (both of which I manually soldered last time), which was far faster, tidier, and easier. I've also got more confidence that visual inspection is enough to tell that the board can be plugged into a Spectrum. I still tested every pin on the memory after assembly for shorts and opens after doing a close inspection with the magnifying glass. I think I can spot any short and most opens now.
My plan is to build the third board by putting the CPLD, memory and associated discrete components on all at once - I think the solder paste will stop the little 0603 parts from blowing away (and when it melts, the surface tension will keep things in place), test this, then do the W5100 end of the board. If that all goes well, and visual inspection is sufficient to see any possible problems, then the fourth board will be done all at once.
The real time sinks at the moment with building the boards is:
- testing for continuity/shorts manually
- picking and placing the passives
As I've already said, I think the first can be eliminated by visual inspection then software testing on an actual Spectrum. The second can be sped up with a sort of 'production line' approach - lining up the tapes of discrete components in order and doing one set at a time.
On the software side, I know there are some code improvements that need to be made to the C library, and I also need to test the C library using caller function call conventions (all functions are callee convention by default, this is where the function itself cleans up the stack, rather than the code that calls it, which saves quite a bit of space).
I also decided the 'major' example program for the Spectranet will be a basic IRC client. I decided on an IRC client because it's something people can use straight away, the protocol isn't complex, and it shows the multiplexing of user input and receiving data from the network. I'm only going to write the very barebones of it (for now), just enough to get on a server and chat, probably nothing like DCC for the moment. This has also let me explore a bit more of the Z88DK, for instance, using the Z88DK's IM2 support for handling the keyboard (which I did today). It's also some code that others can extend if they wish and turn into a fully featured client. Also, the IRC input and screen handling routines can be re-used for other similar types of program, such as a MUD client, or even something like AIM.
But the aim of all of this is to have hopefully half a dozen development boards available for others to get their hands on by the end of the month, in a state that's immediately useful.
Winston 23:02, 8 June 2008 (BST)
Configuration
While it's all fine and dandy to have a working C library, you've also got to be able to configure the Spectranet to get on the network in the first place!
For all my testing, my ethernet uploader program (on the Spectrum) has been just dumping in some static configuration settings into the hardware. Now the hardware can be configured properly. Part of this was writing a user interface (a simple menu based one) to allow the user to enter the usual settings - whether or not to use DHCP, and if not using DHCP, the IP address, netmask, gateway, DNS servers etc. Configuration is stored in the last 256 bytes of the 128k flash chip (page 0x1F). The configuration program therefore can't run from ROM, because trying to execute code from the ROM you're writing to doesn't work, because the CPU fetching instructions will cause the unlock sequence for the flash chip to no longer be the unlock sequence. Therefore, the configuration program must run from RAM. To do development testing, I've just been loading it at address 32768, but once fully integrated it'll be temporarily stored in the Spectranet's fixed workspace page at 0x3000.
Incidentally, all code that accesses main ROM functions, including standard Spectranet ROM code that's not in the main ROM (0x0000-0x0FFF) does so via the indirect call table. This has several useful properties: code doesn't have to be reassembled if the main ROM gets changed (thus changing call addresses), and most of the code will work unchanged with entirely different ethernet hardware, since it never directly accesses hardware, and all it knows about is a set of public function calls. The configuration utility is no different in that respect. The DHCP client also uses the call table. The only higher level code that doesn't is the resolver code for gethosbyname(), but that's because it's part of the main ROM.
When it comes to coding, one thing I find really tedious is writing user interface code (and the most tedious UI coding of all is writing PHP/CGI/ASP scripts for web sites, especially if someone mentions AJAX, but that's a rant for another day). So the configuration UI is simple and functional. At its core is a table based menu generator, which is a table of pointers for menu strings and function call addresses to handle the menu selection. A similar approach is used to display the current configuration settings - a table of pointers to strings, and a table of pointers to the memory that contains the actual configuration option. Doing it this way is more flexible (and I can re-use the menu generator for things like the NMI menu) and a great deal less tedious than writing a bunch of repeating asm code to display various options.
Flash memory is sort of like EPROM, except to re-write it you don't need to shine UV light at at - instead, you give it an erase command. Writing to flash, the only operation you can do is change a bit set at 1 to 0, and not the other way around. The erase operation essentially sets the sector back to all bits set to 1. The flash chip, incidentally, doesn't care if a byte has been written to before - if, for example, you write the value 10101010 to a byte, you can write it again - to 01010101, without needing to erase. The type of flash I'm using is organized into 16k erase sectors, so you don't have to rewrite the whole chip. Even so, 16k must be rewritten. What the configuration program does is copies the last 16k of flash (in which the last 256 bytes contains the configuration area) to the Spectranet's RAM. The configuration program then modifies this copied data in RAM, and when you choose 'Save and exit', it erases the last 16k sector in flash, and copies back the 16k that was copied in earler, thus preserving anything in that last 16k that someone might have put there. It uses Spectranet RAM to do all of these operations, since this won't affect any programs a user might have in the Spectrum's main memory. I want to follow the principle of 'least surprise', and this clearly includes not stomping all over a program that the user might be working on in main RAM! (And in any case, I included static memory on the Spectranet so that the ROM as well as networked programs have some extra workspace).
As far as general progress is concerned, I would have preferred to be a bit further on than I am now, but I'm getting really close to having something suitable for others to do some development with. I have some more solder wick now, so I'll also be making a few boards for development use this month. I also want to write at least one "real" program that does something in the real world for two reasons: provide a useful working code example, and to provide real world testing. I will probably do something reasonably straightforward like a simple IRC client, or perhaps a minimal HTTP server.
Winston 14:56, 1 June 2008 (BST)