Old News (Apr 08 - May 08)
The C library takes more shape...
After a week off in Scotland, it was time to get more of the C library implemented and tested.
The new bits are everything to do with the server side for SOCK_STREAM (i.e. tcp) connections - listen and accept, and multiplexing connections - that is, allowing the programmer to be able to use more than one connection at a time. On Unix, select() is typically used for this job, but more recently, poll() has become available in any Unixy operating system. It's not really necessary to implement equivalent functionality to the Unix/Windows versions of these system calls. In deference to the hardware, therefore, I've seen fit to implement more than one poll function:
- poll_fd - Polls a single socket handle
- pollall - Polls all open sockets
- poll - Pretty close to the unix version - polls a list of sockets
So far, poll_fd and pollall are implemented and has had a bit of testing.
There were a few wrinkles. The socket library hides the hardware from the programmer, giving the BSD socket library. However, the underlying TCP offload engine treats sockets a bit differently from the well-known BSD interface that any Unix or Windows programmer will be familiar with. One such point is the whole listen()/accept() - the underlying TCP offload engine requires you to set up the next socket to allow another incoming connection to the same port. The BSD socket library does this all automatically when you accept() a connection - so the Spectranet socket library does this, too. I ran into something I've never run into on an 8-bit system while testing this - a Heisenbug - one of those bugs where the attempts at observation of the bug makes it go away! However, this one was easy to fix: it seems that the W5100 TCP offload engine needs a substantial pause (well, substantial from the point of the Z80, a few milliseconds) between closing a hardware socket and re-opening it. Timing problems are the cause of many Heisenbugs. I'll have to write to Wiznet and try and find if this time value is defined anywhere; for now I've just put a delay loop in the code when reallocating hardware resources for a listening socket.
It was kind of cool to have four simultaneous TCP connections to a Spectrum though, with the Spectrum handling them all without blocking, as well as checking the keyboard! Not really a huge feat, but fun nonetheless. One of the demo C programs I may write could be a web server for the +3, with pages on the floppy disc drive :-) Or perhaps a version that uses ResiDOS (which should be considerably faster for IDE hardware compared to the +3's FDD)
Next week, I'll order parts to make up more boards since I only have one working Spectranet board. I need more solder wick for one! I also want to make some changes to the CPLD before getting any development boards out - the CPLD implementation could do with some improvement!
Winston 21:18, 22 May 2008 (BST)
First tests of the C library
The C library isn't complete yet (not by a way) but it's complete enough where I could start testing it with a short C program. This revealed a few bugs in the ROM code caused by hardware surprises. For example, the W5100 TCP offload engine doesn't set a source port number when you connect to a remote host. This hasn't shown itself in the past, because all the testing I had done previously was using hardware register areas that had been previously used as a server socket (and therefore, the local port address was set by the prior usage of a server socket). This resulted in an additional system variable so that a reasonably unique port number gets chosen for the local port when connecting to a remote host (it's initially set by the 16 bit pseudo random number generator, and then simply incremented by one with each call to F_connect in the ROM thereafter, or by a mechanism in the F_sendto routine if the local port number is not yet set).
The client test was just to open a new socket, and connect it to spectrum.alioth.net port 80 (i.e http) and do a GET / HTTP/1.0 and print the result on the screen. No, it ain't a web browser (someone else can write that!) but it was a useful test, since it must exercise many functions: socket(), connect(), gethostbyname(), send(), recv() and sockclose().
The pseudorandom number generator (which is part of the ROM library) is an interesting little exercise. This is used to generate transaction IDs for DNS queries, the XID for a sequence of DHCP packets, and the local port number of connections to remote hosts. It's easy to take for granted today the pseudorandom number generators we have on modern operating systems: right from turn on they have a decent amount of entropy to get started on. But a Spectrum, on power up, is almost completely deterministic - by normal means, there's no source of entropy. For instance, it will always take the same number of T-states to reach the reset function in the Spectranet ROM which sets the initial seed. Without having a mass storage device that's guaranteed, there's nowhere to save the seed from previous uses of the PRNG, either - so we can't even use that (and having to rewrite a 16k flash sector to do so is unwieldy, and battery backed RAM would make for a highly significant increase in cost of making the PCB in both terms of the circuit needed and the PCB space for the battery). However, saving graces are that the random number seed doesn't really need to be awfully good for our purposes - it just needs to be good enough to avoid confusion. (Indeed, at a protocol level, a DNS server or DHCP server doesn't give a jot if the transaction ID or XID never changes. However, when debugging it could lead to serious headaches as you go through packet logs not knowing which event caused which sequence of packets).
However, there is one bit of entropy that should be good enough: the state of RAM on power up. RAM doesn't start all set to zero - some cells will be set to 0 and some to 1 on power up for a number of factors, and while you'll see repeated patterns in freshly powered up RAM, there is also a fair bit of randomness of the RAM's state and it varies from machine to machine. It gets better if the Spectranet initialisation routines are running because someone pressed reset - because the system variables will have all done something - FRAMES will have been updated at least. So the PRNG's seed is set by generating a CRC of a few K of the Spectrum's lower RAM, centred over the system variables area, and saving this as the initial seed for the PRNG.
Winston 12:49, 9 May 2008 (BST)
More coding - C library
So I'm figuring out how the guts of the Z88DK works, for that's what the C socket library will be built for. (Hopefully, it'll be incorporated in the Z88DK proper once it's finished).
The C library is actually quite simple - it's really just a wrapper around the ROM entry points to get C function call arguments off the stack and into registers (well, except for the accept function that does a little bit more than the assembler version). I also fixed a few bugs in the ROM code, and got the ROM functions safely into a proper ROM image, rather than the mass of experimental code I was using.
I've done rather less work on the Spectranet than usual, the weather's just been too nice to stay indoors and code (I know I know!) - it's rare we get weather that's actually hot here...
I'm also experimenting with the solder paste to try and discover exactly how to get the right amount on the pads. Yesterday's attempt was a bit of a disaster because I put the CPLD on a new board the wrong way around. I noticed it before removing the hot air, but in moving the chip, it got solder stuck under the chip where I just can't reach it with the solder wick. Then I ran out of solder wick (although not before getting a CPLD onto another board with the hot air and paste).
Next week, I'm away for a week in Scotland so I doubt all that much will get done on the project until I get back.
Winston 19:04, 6 May 2008 (BST)
More coding - a working DHCP client
It bucketed with rain all day Saturday, so there was no better time than to write a DHCP client in Z80 assembly language.
By early afternoon, it was working. Like the guts of the gethostbyname() implementation, it uses the socket library to do its stuff as opposed to directly poking at the hardware. I'm not sure I'm entirely satisfied with what I've written (which is largely the case with anything I write in assembly to be honest), but it does work reliably and it can be improved later. There are some other cases I would like to test, such as testing with two DHCP servers on the network.
I currently have all the basic routines I need to turn the pile of code held together in an experimenter's framework into a useful ROM image, so this is what I'll do next. Incidentally my plans for the ROM are this:
- Core functions in page 0.
Page 0 is fixed in the memory map in the lowest 4K page (0x0000-0x0FFF). The core functions are a routine to do basic initialization, dispatch trapped addresses (i.e. things like RST 8 and maskable interrupts), the socket library, the gethostbyname() routine, and a handful of user interface functions so that other ROM modules can generate stuff on the screen (so the 42 column print routine) and get basic keyboard input from the user.
- Other stuff...elsewhere
On reset or power up, the reset routine will page through each 4K page in the flash chip, and look for a vector table in the first 18 bytes of the page (8 entries, each 16 bits long, then a 16 bit CRC). So you can put a program in any 4K page of flash, and the boot routine will go and look for it. When a valid vector table is found, the boot routine will call the address pointed to by the first entry in this table, where the ROM page can have its own initialization routine. (If it has no initialization to do, then leaving the vector address at 0xFFFF will tell the boot routine to ignore it). At this point a ROM program can register any user interface it has so it gets an entry on the NMI menu.
The DHCP client, W5100 initialization routine and configuration user interface count as 'other stuff' and have the same standing as any ROM program that anyone cares to write. The W5100 init routine, for example, will get called - and check to see what should be done: if the configuration (stored in the last 4K page of flash) says we have a static IP address, configure it - or if it says not, it will invoke the DHCP client. On boot it will also register itself with the NMI menu routine, to give a 'Configure the Spectranet' menu entry. There may be other ROM programs I will write - I'll almost certainly make a ROM based FTP client, for instance. I'd like it to be possible to launch a snapshot file directly from an FTP server as an example.
Winston 22:08, 29 April 2008 (BST)
It's alive!
The board was completed last night, but it didn't work straight off - but it wasn't hard to find the cause: a bad solder joint from the 25 MHz crystal to the W5100, and a bad joint on the 74HCT245; just a simple case of inexperience when using solder paste and hot air for assembly. The bad joints were easily fixed - since there was already solder on the pads, it just required pressing the tip of the soldering iron on the pad and offending pin to remelt the solder and properly make the joint. I decided to leave it until today to do the work though since it had gone midnight - and last time I tried to fix stuff gone midnight I ended up irretrievably breaking it by ripping up a pad.
A couple of observations:
The paste is great for doing chips with hot air. It's so much better than using the soldering iron to solder fine pitch surface mount. The paste is terrible for mounting the small passives with a soldering iron (and hot air would blow 0603 components away).
So without using a reflow oven, the best assembly method is do the chips with paste and hot air, then do the discretes with the soldering iron.
Getting back to the board - after fixing the two bad solder joints, I powered up the Spectrum and the experimental ROM code showed a good read back of the configuration data sent to the chip, and the LINK light came on as the ethernet connection was detected. A quick test showed everything working nominally - I sent a .SCR image of the Elite loading screen over the network, and it appeared on the Spectrum's display.
So what next?
First, give the prototype board a good workout (and disassemble the breadboard prototype, to reclaim my breadboard and some desk space). Then, get a basically functional ROM in place that exposes the socket library, plus a C interface. By basically functional, I mean:
- DNS lookups.
- TCP (SOCK_STREAM) and UDP (SOCK_DGRAM) socket interface for asm and C, with a reasonable amount of testing.
- DHCP client.
- Configuration interface.
While waiting for various bits to arrive to complete the first PCB, I also mulled over the current CPLD design, which is my first use of programmable logic. I think there are quite a few improvements that need to be made here, so things like the paging mechanism may still change yet (which will cause a code change).
I *just* missed the 26th birthday of the Spectrum for getting the first prototype PCB up and working - so I've overrun by a couple of days than my intended schedule. I hope I can have a few prototypes made in about 6 weeks time which are suitable to get into the hands of developers. Overall, the PCB itself has gone pretty smoothly - only one error on the prototype PCB, and it was only a minor one that was easily worked around. Well, so long as nothing turns up when testing the through port!
Winston 20:46, 25 April 2008 (BST)
Grey goo
I received the bits I need to continue soldering from Farnell today, namely soldering iron bits, and some solder paste (Edsyn CR44, in a syringe). This time, things went rather better. I decided to start with a fresh PCB for now and go back to the others later.
I've never used solder paste before. You can get it in bulk (in tubs) or in syringes - and it's fantastically expensive. A small syringe of the stuff is 10 pounds! However, you only need to use minute amounts. In fact, even though I put what I thought was a minute amount on the pads (I gave up trying to put individual blobs on each pad, and instead drew a line of the stuff down each row of PCB pads and hoped surface tension would do the rest) it was too much. However, it's neater than hand soldering fine pitch SMD components, and there's far fewer solder bridges, so less mess left from wicking up excess solder. This also saved me from having to test each pin on each chip, since I could easily see how much neater the solder joints were, and see any problems with a magnifying glass.
The solder is a kind of grey goo - a viscous paste which is a thick suspension of solder and flux in some kind of solvent. After laying down a line of this on each row of pads, I placed the chip (first the flash), and then got my trusty B&Q heat gun - which on its lowest setting only happens to be nominally the correct temperature for reflowing solder! The air stream isn't too violent either, so the chip doesn't get blown away. It's still too much for tiny 0603 size discrete components, though, so I hand soldered those.
I should have used paste for the RAM, but seeing as it's in a SOJ package, I thought it would be just as easy to hand solder...but it ended up taking far longer than the CPLD or flash chip in the end.
The good news is that the problems I was having earlier must have been just a bad chip. This time, the Spectrum came up just fine. I've run some test code and even programmed the flash, and it all appears to work nominally. The only thing left to do on this particular board is mount the W5100 itself and its associated buffer IC.
Winston 00:03, 24 April 2008 (BST)
Frustration!
The prototype PCBs arrived on Thursday, and with the excitement of a little kid, I unpacked them and got to working on them. First, I attached the CPLD and the 3.3v supply components, decoupling capacitors for the CPLD, and the pin header for the JTAG lead. The CPLD programmed no problem. So I then added the flash and RAM, and associated discrete components for the ROMCS circuit.
It all started going downhill from there. First, I got a screen of vertical stripes (which was expected) - the flash at this point contains 0xFF, and this is the RST #38 instruction, so the Spectrum would just execute RST #38 over and over and the stack eventually overwrites the screen, hence the vertical stripes. I then closed the 'disable traps' jumper... and got vertical stripes. I discovered that despite triple checking the board, the jumper was connecting the CPLD pin to ground instead of Vcc. An easily worked around problem - just go into Xilinx ISE and change the CPLD to prevent traps.
This still didn't fix it, instead I got random garbage on the screen. After much fiddling and testing I got no further, and could find no errors on the PCB that would cause this. So I set about building a second one. First, I put the CPLD on the board, and configured it. Then the ROMCS components. So far so good, and the relevant OUT commands from BASIC would crash the machine (which it should at this stage). Then I put the RAM on the board, and POKEd in a short program to test I could write a byte and read it back; this worked. Then I wrote a small test program to test memory paging (I also wrote a bin2tzx utility that can be found here in SVN [1] so I could send freshly assembled programs to the Spectrum via the tape socket). This all worked satisfactorily.
Then this morning, I set about putting a flash chip on the new board. However, the solder for some reason wasn't flowing well... and then the tip of my soldering iron broke off just after I got done soldering one side and was cleaning it up with solder wick. I was sure I had a spare soldering iron bit, but it was not to be.
So I'm left with a board that's non-functional for reasons still unknown, and a second board with a half-soldered flash chip and a broken soldering iron bit, and I suspect due to the poor solder flow I'll have to 'hot air' the flash chip off the board and start again. Hrmpfh. I had hoped to have the first board completed by last night.
I don't really feel like working on the software today, so I think I'll take the rest of the day off. Perhaps I'll improve the bin2tzx utility a bit and submit it to WOS for the utilities section. I'll also order some solder paste and needles from Farnell. While the big square chips (CPLD and W5100) and the RAM isn't too bad to hand solder, owing to the long pads on the PCB, the flash chip is really annoying to hand solder, and I think paste and hot air will be ten times easier.
It's of course not all bad news - I do at least know that the CPLD and RAM are working correctly, as is the reset circuit and ROMCS circuit (for both 48K and +3/+2a) is working correctly. I've also checked the flash chip against the datasheet again and found that the connections are all correct. At the moment I'm suspecting either a "poor short" that the continuity tester on my multimeter won't beep at but which is causing problems, or a bad flash chip. Unfortunately, I'm not going to be able to do anything more to the hardware until the middle of next week when the parts I need arrive from Farnell.
Update: I hot-aired off the flash chip from the nonfunctional board, and it no longer crashed the Spectrum and the static RAM read/wrote OK, so there was the culprit. It's either a duff chip, or it's not TTL-compatible (which it *should* be, but there's no way of telling from the package markings and the datasheet gives DC characteristics for both TTL-compatible and CMOS-only devices). If the second board fails in the same way when the flash is attached, I suspect the chip's not TTL-compatible, and I'll have to order alternative flash chips.
Winston 11:21, 20 April 2008 (BST)
PCBs ready...
I just got notification from the PCB fabricator that the PCBs are done and have been sent...
In the meantime, I've not got all that much done, various other projects (including fixing a backed up drain) and social engagements mean I've spent very little time on the code over the last week. However, I now have a working input routine which will work for the user entering configuration items, like the IP address, netmask etc. I've also written (but not yet tested) some routines for saving configuration information into the flash chip. I need to tie it all together with a simple menu-based user interface. With luck I can get that done this week. However, when the PCBs arrive, I expect to spend time on assembling those, too. Busy times ahead!
Winston 22:45, 14 April 2008 (BST)
Back to Code
Firstly, the PCB layout has gone to the fab to be manufactured...
Not the fab of my choice. Unfortunately, I didn't notice that PCB Train had a design rule about drill holes (the drill hole must be at least 0.5mm from an inner layer copper trace). There are parts of the PCB where I've had to squeeze in a lot of vias and thread tracks between them on layers 2 and 3. Oops. So it'd either be a week of redesigning, or use a different fab. I wanted to use PCB Train because it's in the same timezone and run by people who natively speak the same language as me. Anyway, a fab has been found (pcbcart), and a random search of the web shows that others who have used them have given them glowing reports. They also do different coloured solder masks, so I don't have to use the bog-standard green. I went for blue for the prototypes. (In the past I've used Olimex in Bulgaria, but they only do 2 layer eurocards and they don't go down to the design rules needed for 0.4mm pin pitch. I also looked at a few others in the USA, but that would have added lots of delivery time because all the ones I saw that do 4 layer along with solder mask and silkscreen ship it off to China, so the board would go China -> USA -> here). PCB Pool in Germany are just far too expensive - they charge you extra for using industry standard Gerber files! They seem to only really cater for Eagle users. Pcbcart look like a good bet - they will also do odd shapes and 'gold fingers' on request - so if I use them for the production boards, they can put the through port cut-out so I don't have to do it, and also gold plate the through connector. I've opted for no 'gold fingers' on the through connector because I want to see how much the default finish lasts as an edge connector - the gold plating almost doubles the cost of the PCBs. Sinclair got away with a tin-plated edge connector, wobble notwithstanding!
So, back to the code. This weekend adds the following bits of functionality:
- gethosbyname - actually, a subset - it only returns one address rather than a list and all the other hostent stuff. But that covers 99.99% of all situations for networked programs on unix and Windows, so it should be enough.
- keyboard input - internal functions to be used by the configuration UI
- screen output - the existing 42 column routine and extra routines around it
The gethostbyname implementation just needed to use the DNS lookup routine I wrote a couple of weeks back, as well as new routines to be able to parse an IP address as a string - if you pass an IP address to gethostbyname, it won't do a lookup, rather, it will convert the string to a big endian 4 byte integer that represents the IP address. I also added a function to convert a 4 byte big endian integer to a dotted decimal string IP address. The 8 bit int-to-ascii converter is a bit primitive at the moment, for instance, it will display 1 as 001, but it will do for now.
Keyboard input required me to do a bit of research. Not wishing to re-invent the wheel, it uses the CALLBAS exit in my ROM to call the keyboard handling routines in the Spectrum's BASIC ROM. This is fine on a 48k machine, however, on a 128k machine, on reset (when the Spectranet code is active, but before the ZX ROM has been initialized), the wrong Spectrum ROM is paged in (the BASIC editor), so routines to initialize the 128k paging environment vars, and page in the right ROM were added. The get key routine just calls the ZX's ROM KEY_SCAN, K_KTEST and K_DECODE to turn what the user did into the right ASCII character. The +3 added an extra wrinkle as there's an additional page register to prod (plus its associated system variable) so the routine crashed the first time I tried it on a +3. But it works now.
Since the screen output must work before the ZX ROM has been initialized, I'm using the 42-column routine that I wrote ages ago (it's used in the diagnostics board I made last April, since that board has to assume a non-functioning ROM until it's been tested). In fact, sharp eyed readers might have spotted the DNS test from a couple of weeks back wasn't displaying the output in the normal Spectrum character set. While I could write code to get enough initialized to use the ZX ROM's RST 16 character print routine, I would rather use a 42-column character generator for the Spectranet since it allows a bit more on the screen.
I need to generally learn a bit more about the ZX ROM, there's quite a few routines I could use that will save me having to write code. In any case, there's the vexed issue of getting the parser to work; I'll have to read through the stuff Garry Lancaster sent to me.
Plans for the time until the PCBs arrive:
- Get a basic user interface going to allow for configuration.
- Finish off the DNS 'A' record lookup routine - code in timeouts, and failover to second/third DNS servers.
- Perhaps look at writing the DHCP client.
Winston 20:55, 6 April 2008 (BST)