Old News (Dec 07 - Jan 08)
The basic socket library
So with the hardware (mostly) working, the next step is to write the basic socket library - first for assembler and C. As already mentioned, it's my intent to broadly follow the way the BSD socket library works, since that's really the standard. Even Microsoft didn't re-invent the wheel on that one, they provide the same interface. People are familiar with it, it's flexible that it works with every piece of hardware in the world. When I say "broadly follow", I'm aiming for a subset of the library (particularly for the asm version). The C version will have the same function prototypes as the real thing. By contrast the asm version will have the same functions (socket, listen, connect, bind, send, recv... etc) but with some parameters missing. For example, the family argument to socket is irrelevant to the asm version - an asm program isn't portable anyway, and the socket library only understands AF_INET sockets, so there's no point using the family argument - and people using the asm interface rather than the C one probably care enough about performance that checking useless arguments will be an impediment.
The rationale for modelling the BSD interface for asm (and replicating it for the Z88DK) rather than simply wrapping the W5100's hardware features is that if the hardware changes, then software doesn't have to change too. Or even the TCP/IP stack changing (you don't have to use the W5100's TCP offload, you can use your own stack running on the Z80).
The other part of the abstraction is to provide a jump table for all the ROM functions. At the cost of a dozen or so CPU cycles, the table does two things:
- the entry point causes the ROM to get paged in.
- the call table means that programs that depend on these functions don't have to be recompiled/reassembled when ROM locations inevitably change.
What about BASIC though?
First, I want to get the asm socket library thoroughly debugged since it's the basis for everything else. However, I think BASIC support is actually pretty important, since it lets any Spectrum fan have a bit of fun with networking. Ever since the days of the VTX-5000, I've known how much having a network transforms a computer... and I'd like it to be available to all, including people who aren't hardcore programmers.
Incidentally, having sockets available from BASIC will I think make the Spectrum quite a handy embedded platform for the hobbyist - reasonably easy to program, and easy to make homebrew electronics for. With the built in BASIC interpreter, it makes experimentation very easy. Having network access as well just makes it that much more useful.
I do have to find out how the BASIC parser works, but World of Spectrum has some useful resources for that such as the Complete Spectrum ROM disassembly, as well as the Shadow ROM disassembly - which has examples of how to do BASIC extensions.
The BASIC extensions probably won't just be providing socket, listen, accept, send, recv... etc - rather combining some of the calls - a bit like how Perl's IO::Socket::INET allows you to open a socket to a remote host with a single call. BASIC programmers likely don't need the flexibility of keeping all these calls separate - instead, they likely want ease of use (or they'd be using asm or C!) So I plan to have a simple connection possible using no more than 4 keywords - one to open, one to read, one to write, one to close.
Further hardware prodding
While testing out the flash programmer, I've found that the 0xFF problem (mentioned last week) hasn't really gone away altogether. I discovered that the board was working rather slowly. Just for fun, I sent some game SCREEN$ images to the Spectrum over the network, using the program I wrote for loading programs over the network - it takes a very simple header containing memory location and length, so I could just send data with the location 16384 to send a SCREEN$. It loaded rather slowly I thought (a throughput at a guesstimate of 8k per second, which is appallingly slow). So digging through the millions of hardware registers for the W5100, I found that by default the device uses a "delayed ACK" option, which instead of acking packets as soon as they are successfully received and stored in the rx buffer, waits for a specified time before sending the ACK. What useful purpose this serves, I do not know, and it's baffling that it's apparently the default.
So I turned the ACK delay off, which resulted in SCREEN$ images loading instantly. However, the old 0xFF byte problem started again, where bytes in the buffer set to 0xFF get read back as random values. Unfortunately, if I use a smaller pull-up resistor value than 4k7, it loads the Spectrum's bus too much, and all sorts of funny things happen to lower RAM. The Spectrum has a very simple bus multiplexer - just a bunch of resistors across the bus so the ULA can read off lower 16k at the same time as the Z80 reads off the upper 32k. And putting pull up resistors on the data bus causes this not to work properly, unless the pull-ups are rather weak.
This didn't stop me reprogramming the flash chip for the first time though - the program doesn't contain long strings of 0xFF bytes, so even with this problem the breadboard prototype can be used for this.
So - really, there's nothing for it - the W5100 is going to have to be buffered since it can't drive the Spectrum's bus properly. I put one in the CPLD design, and it uses an amazingly huge number of macrocells (about 20 for a bidirectional tristate buffer!) While I have quite a few macrocells left over at the moment, I would rather save them for things that would take lots of chips to implement. A suitable bidirectional buffer, on the other hand, is a single 74-series chip. So I may use a separate chip for the buffer. It's entirely possible that the W5100 won't have this problem when put on a proper PCB with a decent ground plane, rather than my breakout board - I can put vias under the chip for instance and give the chip a much better path to ground than it has at the moment. Also, I can get decoupling capacitors much closer to the power pins. The only trouble is I have to actually get a board made to try this out and this is expensive!
And finally...
I'm away for two and a half weeks, so there probably won't be all that many updates during this time since I don't have access to the hardware! I did work a bit on the socket library on the plane though. Probably the only person on the planet to be writing Z80 asm five miles above an ocean!
Winston 18:31, 31 January 2008 (UTC)
Sending programs over ethernet
The first application of the prototype is to program itself.
It's a bit of a pain to have to dig out the 128k flash rom each time I want to try out some new code. So I've made a self contained standalone program (which incidentally is NOT an exemplar of how the board's programming interfaces will work - it's very rough and ready) which allows me to load a program to an arbitrary address in the Spectrum's memory and jump to it, or write that transferred data to pages 0 to 3 of the 128k flash chip (although I still need to add the bit that actually programs the flash). I also made a short perl script to send the file to the Spectrum. The server (on the Spectrum) listens on a TCP socket. The first 4 bytes contain the destination address in RAM and the length of the data. The rest of the data is the program or data being sent. The program is in the Subversion repository here ( [1] ), with the corresponding perl script here ( [2] ).
This should save a huge amount of time when testing things out on the hardware.
Incidentally, I've not seen a recurrence of bad register reads (yet) where the register contains 0xFF since adding the pull up resistors, so I may have been imagining it. But I'm going to keep a very close eye on the netmask readback on bootup. If the problem really hasn't completely gone away I can always buffer the W5100 via the CPLD, since I do have some spare macrocells.
I've also been adding the remaining functionality to the CPLD, adding circuits to allow the readback of certain state data. If there's enough macrocells remaining I may also make the memory pager registers read/write instead of write only since it'll give a little bit of a speedup to the paging code (since there will be no need to save the current page in memory).
Winston 23:31, 22 January 2008 (UTC)
First packet!
The Spectranet just sent and received its first packets!
After wiring in the W5100 ethernet chip, I added the code that copies to and from the network buffers and pokes the appropriate hardware registers. It's not as straightforward as LDIRing a bunch of data into a buffer - you have to read some (big endian) registers to calculate an offset, and then check the buffer doesn't wrap - the buffers are circular. Then either LDIR it to the right place, or if it wraps around, LDIR the first part, and then set everything up to continue from the bottom of the buffer. Then poke a few more registers to tell the hardware how much you've just done, and you're away.
The code update of interest today, therefore, will the file w5100buffer.asm over here ( [3] ) - well, all of the w5100*.asm files really.
It's quite a major milestone - I now have a rubber key Spectrum that talks over ethernet (somewhat). But as I keep saying, don't hold your breath: sending some packets and receiving some packets is still a long way off from having good general purpose code so that other people can use it, and other major tasks must be done - such as settle on the final hardware design, draw out the schematics, and turn this into a working PCB layout.
However, there's also some bad news (although not as bad as I initially thought).
The W5100 doesn't seem to be copying the value 0xFF correctly. If, for instance, you send a bunch of data which contains just the value 0xFF repeated, when the Spectrum reads it from the ethernet buffer, roughly every 8 to 15 bytes or so, it'll get a random value instead of 0xFF. It doesn't happen with 0xFE, 0x7F (or any other value with lots of 1s) - only 0xFF. This includes register reads - so for instance, if you read back the netmask, sometimes you get something other than 0xFFFFFF00 (although for some reason, it's less random when reading back registers instead of buffers).
Either the W5100 has a bug (unlikely, but it has been known: and Wiznet do have some errata on the chip, for instance some revisions of the chip run extremely hot), or it is suffering from ground bounce (that is, when a chip with high speed outputs tries to change a lot of them at once, it momentarily draws a relatively massive current that drags the ground up above 0 volts). If it's the former... it's game over. If it's the latter, it's just a PCB layout problem that I can fix.
Unfortunately, there is only one forum that discusses the W5100. In the past I've contacted Wiznet's tech support over other issues with the chip, but English isn't their first language and I don't think I was effectively communicating with them. Also, I do not have a digital storage scope so finding out whether it's ground bounce or not will be very hard since it's a brief, transient problem. Once my username request is approved for the forum I'll try asking there - see if anyone else is using this chip on a 2 layer breakout board and has the same problems. Or perhaps they never noticed: many applications will never need to receive the value 0xFF, even less a string of them - and a problem like this would go undetected. (Had I not noticed the netmask readback being incorrect, I'd have never noticed either - and would probably right now be going berserk trying to find out why the new ROM image I transferred over ethernet had strange bugs - the chip had after all worked perfectly with the text data that I sent to test out the receive data routine).
Update: Ground bounce, or at least trouble driving a long bus with lots of parasitic capcitance is almost certainly the problem: adding 4k7 pull up resistors to each line on the data bus made a significant improvement (data from the transfer buffers is now correctly received, regardless of the amount of 0xFF bytes, but there's still problems reading the netmask back).
Winston 17:52, 20 January 2008 (UTC)
LNK... FDX... 100M...
A quick update: I made up some breadboard cables, and connected the W5100 module to the setup. Aside from the breadboard now resembling a plate of spaghetti (I really ought to talk to Chris Smith of Harlequin fame about neat breadboarding!), with much anticipation, I powered the Spectrum up. When I released the RESET pin, to my relief, it came up, and appeared to work as a Spectrum should - and the LNK, FDX and 100M lights lit up on the W5100 ethernet module.
I've not got as far as actually doing anything with it, merely ensuring it powers up and doesn't cause bus loading problems. The next task will be sending the Spectranet's first packet, and then make a simple program that allows me to flash the ROM with the Spectrum, over ethernet, so I'm not having to dig the flash chip out from underneath its mass of wiring each time I need to update the ROM.
Incidentally, a PCB layout for the W5100 module can be found here ( [4] ) in Subversion, if you fancy building your own W5100 breakout board. It's pretty much my translation of the Wiznet reference schematic to a 2-layer PCB that's optimized for home etching. (You'll need gEDA PCB. I do intend to put PostScript and Gerber versions of PCBs on the wiki in the near future).
Winston 23:16, 16 January 2008 (UTC)
Testing the new CPLD
I had been planning to work on other projects this weekend, and not do anything at all with the Spectrum ethernet prototype... but I shot myself in the foot with the other project (due to sloppy thinking that 5 minutes careful thought would have prevented), so I worked on the ethernet project instead.
The goal: test the new CPLD works at all; it's a 3.3 volt chip with 5v tolerant I/O. Eveything in the Spectrum should be TTL compatible (so 3.3v is a perfectly valid logic 1), as should be typical 5 volt CMOS flash and static RAM. However, there's nothing worse than spending hours making a PCB then discovering it can't work (ever), so to the breadboard it was.
I had actually wired up half the circuit in the week, then finished it off today.
It didn't work. The usual random colour attribute vomit on the screen when I powered up, which is so familiar to anyone who has either a Spectrum with bad lower RAM, or has just plugged in a faulty peripheral. So I unplugged, and checked the Spectrum still worked (which it did). I then removed the memory and the connection to the ROMCS level shifter so only the CPLD was connected - with the same result. The CPLD was powered and apparently working, so I unwired everything - sure I had a short. I rewired, and tested the Spectrum at each major checkpoint (first the address bus; OK - then the data bus; OK - then the control lines - vomit). I started removing connections to the control lines (all of them CPLD inputs - nothing actually driving a signal into the Spectrum yet), and it started to work as soon as I removed the RESET connection.
It turns out that the RESET line gets stuck at about 0.2 volts when connected to the RESET input on the CPLD. If you momentarily touch RESET to +5v, it un-sticks and everything works just fine. Testing the voltage level when RESET is high shows that it's sitting at 3.3v, which suggests the 5v tolerant I/O on the CPLD is done by clamping the input with a zener diode - but there must be something more to it if it causes RESET to stick low (if the 5v tolerance is just an on-chip zener and resistor, it shouldn't affect stuff until the voltage exceeds 3.3v). Still, the input impedance must be pretty high since the other I/O lines don't bother the NMOS circuits in the Spectrum. I will have to examine the Issue 4 Spectrum schematic to find out why the CPLD causes it to stick low. The fix (for now) is to add a 4k7 pull-up resistor to the RESET line - it functions fine like that.
With that mystery solved, I added the flash ROM and 128k static RAM back to the circuit. I also modified the routine I've been testing the hardware with, so it resides entirely in page 0 of flash (which is permanently mapped to 0x0000-0x0FFF when our memory is paged in), and the workspace for the character printing routine into our RAM, instead of upper RAM on the Spectrum. More testing is needed (such as the paging mechanism for the memory map between 0x1000 and 0x2FFF) but that can wait until tomorrow. Test code can be viewed here ( [5] ) - I use sjasmplus, and 'testtraps.asm' assembles all.
I also did some rudimentary compatibility testing. First, with my own diagnostics board since this is dirt simple and hopefully I can understand it if it goes wrong. Compatibility with downstream devices with their own shadow ROMs is peformed by nailing A15 high on the through port whenever we're paged in (which stops anything on the through port mapping its memory, since it will see an address >= 0x8000, rather than one in the lower 16k). This went fine - the RESET routine running from the flash on the breadboard, and then the diagnostics card getting paged in and running its tests (which all completed successfully). I tried the same thing with the DivIDE - again, it worked fine, the RESET routine running from my flash, and then the DivIDE getting initialized as normal.
I've also run out of jump wires for breadboarding (or at least, ones of a useful length). It seems a bit odd since I breadboarded an entire Z80 based computer using the wires I have now, but somehow I've managed to use nearly all of them. However, once I've done more complete testing on the memory and paging, it'll be time to plug in the W5100.
But the rubber keyed Speccy in the pictures is now a 304k Spectrum, 128k of that being non-volatile. Just imagine what a huge deal that would have been in 1986! Pity NOR flash hadn't been invented then.
Winston 20:08, 13 January 2008 (UTC)
Major breadboarding session on the way...
I have the basic CPLD design now more or less done and simulated within Xilinx ISE, all it needs to do is actually work. So I have some pin-thru-hole flash (Am29F010) and static RAM (Mitsubishi M5M51008B), both 128kbyte, for testing the memory. The current CPLD design (which includes the pager and instruction trapper) fits easily into the XC9572 with many macrocells free. I still have a bit more to add (software enable/disable RST 8 trapping, NMI through, soft enable/disable W5100 interrupts) but there should be some macrocells left over for 'luxury items'.
More on the memory map and paging mechanism on the Memory page.
My plan is to test with just the flash and RAM at first, then add the W5100 breakout board, which should be a simple matter of connecting it to the data/address bus and writing a little bit of code to set it up. While theoretically the circuit is just a straightforward adding-memory-to-the-bus type of affair, I know from my testing of the W5100 with my ALIAC-2 Z80 based computer is that the W5100 seems to load the bus. The Spectrum's NMOS parts just aren't going to be as good at driving heavy loads as the ALIAC-2's CMOS parts, so it remains to be seen whether all the signal lines to the W5100 will need buffering.
Winston 22:41, 5 January 2008 (UTC)
Back to work for the new year
What with the usual holiday festivities, I've not really done much on the project since making the breakout board (except testing the board actually works), and doing a bit of diagnostic work on a new dead 48K Spectrum+ that someone sent me.
I'm getting close to the design being settled for the memory map. The current intention is to do the following: treat all memory (including the W5100) as 4K pages, with 2 pages fixed in the memory map (so when an event happens, we're guaranteed to have a certain page of flash ROM and a certain page of workspace mapped in), and 2 pages that can be mapped from any 4K page of RAM. The memory map will look like this, when mapped into the Spectrum's lower 16K:
- 0x0000 - 0x0FFF - page 0 of flash ROM. Contains initialization code, socket library, various low level function calls.
- 0x1000 - 0x1FFF - Any 4K page of any memory.
- 0x2000 - 0x2FFF - Any 4K page of any memory.
- 0x3000 - 0x3FFF - page 0 of the onboard static RAM. Workspace, system variables, jump table.
The only decision left is this: To allow room for future expansion, the registers for the pagable areas are all 8 bit (allowing for up to 8 megabit chips - i.e. 1MB) to be used, with a 4 bit chip select register (so up to 4 memory chips - three used at present; the flash, W5100 and RAM). This requires two 8 bit registers and a 4 bit register. Alternately, I could delete the 4 bit chip select register, and generate the chip selects from the upper 2 bits of the two page registers. This would allow for up to 2 megabit chips (256K per chip). It would save some CPLD macrocells, and use one less I/O port (so only 2 I/O ports would be required).
I decided to use multiplexers in the CPLD rather than tristate buffers for the output of these registers onto the bus, since this will mean the paging address lines and chip selects would never go into high impedance state (eliminating the need for a handful of pull up resistors). The "upper bus" is never contended with any other device so there's no need for those lines to ever go high-Z.
Winston 12:23, 3 January 2008 (UTC)
More prototyping tools: XC9572 breakout board
Christmas Eve afternoon was spent making the breakout board shown in the photo, and I had enough time before Christmas dinner to check that the board at least programmed correctly (which it did), so I don't think I screwed anything up too badly :-) This is the chip I intend to use as the CPLD on the ethernet board. It has more pins than I actually need, but the next size down is the (more expensive!) PLCC, and then to the 44 pin packages which don't have enough pins.
Apart from pin headers/SIL strips, everything is surface mount, using the passives I intend to use for the final board. The 3.3v power circuit (using a 3.3v low dropout regulator, no heatsink required since the current draw won't be that high) is also the one I intend to use on the final board. Decoupling capacitors are all 0603 sized which is as small as easily hand solderable. I want to get the passives down as small as possible so they don't get in the way too much when routing traces; 1206 is just a bit too big (and gave me a real headache when laying out the W5100 breakout board).
I probably won't get much more done on the project over the next week or two (too many Christmas/New Year type events going on) but hopefully by the end of the month I'll have the W5100, CPLD and memory all working on breadboard and will have sent the first packet.
Winston 10:28, 25 December 2007 (UTC)
Welcome to the wiki
In order to make it easy to document ongoing hardware projects for the Sinclair Spectrum, I've put this wiki into service. At the moment, it can only be edited by me... but in future, I hope there will be others interested in collaborating on the projects here, and MediaWiki makes it easy for others to edit, too.
Onto the most recent news. I've just received my order from Farnell for most of the parts needed to make the prototype ethernet boards. I intend to make a small number (about 8 or so) to allow me to develop ways to economically build boards (economic not just in money, but in time). Experience of hand-making a couple of diagnostics boards showed that this was a non-starter (it took FAR too long - a couple of hours per board at least to assemble!) so this time around, I've decided to use surface mount parts, and build my own reflow system (with a hotplate or a toaster oven - nothing expensive) to try and make the boards reasonably quick to assemble. I also intend to make a handful of prototypes, so I can get a few out to some Spectrum enthusiasts who want to be on the bleeding edge (!) and shake out any bugs.
But in this consignment, I have:
- 10 Xilinx XC9572 CPLDs, in 100 pin TQFP packages. 3.3 volt types (which have 5v tolerant I/O)
- 10 Am29F010 90ns 128kbyte flash memories. This is to hold the ROM code, and a set of applications to go with the board. These are in TSSOP packages.
- 10 IDT71024 128kbyte static memories. Extra workspace for networked applications. These are in SOJ (small outline, J-lead) packages.
- 3.3v linear regulator - to power the W5100 and the XC9572.
- Various surface mount passives (lots of capacitors, a few resistors, a few inductors).
- A couple of tools to make it a bit easier to do surface mount stuff (tweezers, nothing fancy)
- On back-order - 10 Tyco Mag45 RJ-45 sockets with integrated magnetics.
I already have plenty of some of the other parts - lots of 25MHz crystals, enough P and N channel discrete MOSFETs (for the ROMCS hold circuit), indicator LEDs etc.
I still need to get some card edge connectors, LaesQ suggested a suitable supplier (where he gets the connectors for the DivIDE).
The current plan of attack is to make a breakout board for one of the XC9572s, including a 3.3v regulation circuit, and breadboard a test circuit which will include a (DIP packaged) Am29F010, as well as a 1mbit static RAM, and the W5100 breakout board I built earlier. This will enable me to test that the Spectrum's bus doesn't get loaded too heavily (I've already noted that the W5100 puts some loading on the bus from earlier tests), and that the current draw isn't too high; the W5100 can use up to 180mA with the PHY at full power. Once any hardware issues are sorted out, then design the schematics/PCB for the first version of the board... then have PCB Train make 10 of them.
Winston 22:51, 21 December 2007 (UTC)