Linux Home
Automation
(Last updated: Saturday February 24, 2007)
Automation
(Last updated: Saturday February 24, 2007)
mTW523 - The modified TW523 project
The Modified TW523. - Intro -- Introduction -- A word about safety -- tools - Theory -- Layer 1 the voltages and frequency -- Layer 2 the half bit, bits, packets and errors. --- Standard packets --- Dim/Brights packets --- Extended packets -- My intention (theory) - Hardware -- TW523 --- Default Operation --- Modified Operation - Software -- PIC -- PC -- Future additions
-- intro Here's a project that started out as a way to see the bits of X10 traverse the AC and as a way for me to brush up on the electronics that I really haven't used since I left college in 1987. So I've kept it relatively simple so students should be able to follow along and people who haven't done this in a while can say 'Oh, yeah, I remember that.'. OK, now I've been playing with X10 going way back to the days of the brown BSR units. Back then I had the Heath transmitter kit (similar to X10's CP290) connected to to my Atari 520ST and then my AT&T 3B1 (a Unix PC). X10 was a mysterious protocol and back then I just wanted it to work. As the years rolled by I became more interested in how X10 worked and if I could build an controller to communicate with X10 devices. I read the X10 Tech note and understood how the on and off commands worked but the description of the other commands left something to be desired. In addition the TW523 could transmit any command but couldn't receive all the commands. With X10's CM11A the TW523 seems like some what of a kludge. Now if only X10 would do something about it's quality control. The good news is that on the newsgroup: comp.home.automation, many of the regulars have worked out to fix things and how to improve the performance. The motivation behind this is simply that I wanted to see all the bits and the noise. The current TW523 couldn't do it and the CM11A masks the protocol with another layer of software. The PowerLinc wasn't available at the time and I'm not sure how well it works. There have been reports of problems with the powerlincs. So I went about modifying the TW523 so it would pass everything. The intelligence would be added to a microcontroller connected to the outside of the opto-coupler. This would make it safe for the microcontroller to operate. I had thought of using a uController inside the TW523 but I decided against it because of safety when debugging. Remember that I doing this from home where my lab is made up of inexpensive tools that I've gathered over the years. I'm currently using Microchip's ICD (got it during the $99 sale). Having that in the middle of the of a hot ground would be very bad for the PC to which it was connected to. [ Need picture of 'hot' ground. ] - A word about safety. One of the things to note about this circuit is that it is not isolated from the live AC. That bears repeating: Warning system ground is not isolated from AC neutral!!! I once worked with an engineer whose analog was weak. In his defense he strengths are digital and assembly language. He could write assemble like many good programmers can write in high level langauges. One day while teaching a class on basic electronics he took the probe of the oscilliscope and stuck the end into a near by AC outlet. In the process the scope died in a spectacular puff of smoke and noise. The following day the engineer related his story to me. I in turn explained to him that what he had done was to short the AC to ground. I further explained that when ever working with older scopes and AC that an isolation transformer should always be used. Yes I have one directly connected to the HP scope I'm currently using. Now this wasn't to belittle engineers but rather point out that sometime those who should know may not be ware of the dangers -- Tools Tektronics 2230 100 MHz digital storage oscilloscope. Bitscope. bread board [ need picture of bread board ] - Theory -- Layer 1 the voltages and frequency.Theory X10 was design in Scotland where 50Hz is common In the US we have 60Hz X10 basically sends a 120K Hz spurt within 50us of zero crossing of the AC. The zero crossing is where the AC's voltage is 0. A data bit is sent on the first half of the AC cycle and it's compliment is sent on the second half. It doesn't matter if the signal starts out rising or falling as long as there have been 6 ZC's where no data has appeared. Starting with what has been transmitted onto the AC we have the following: The transmitter transmits a 120KHz pulse starting up to 50uS after the Zero Crossing (ZC), the pulse is 1mS in duration (we will ignore the other 2 pulses, which correspond with the second and third phase, for now). An X10 message consists of at least 2 halves. cEach half is 22 bits long (most messeages, there are exceptions). The first half (ID) is sent twice, then a pause of 6 ZC's then the second half (CMD) is sent twice. The transmitter must wait at least 6 ZC's before starting to transmit. The entire message takes 94 bits to send, plus wait 6 ZC's before starting, grand total 100 ZC's. Strange how this corresponds exactly to 50Hz (100 ZC's). The TW523 will send the signal to the end user device as it starts to see the second half of the command. I currently don't know what it does if the second half of the command has bit errors. The original TW523 can not see Dims, Brights, Extended Commands or Extended Data because none of these wait for the 6 ZC's between commands. [ need picture of TW523's proper output] A single X10 command such as turn device A1 on takes a grand total of 50 cycles. For those in Europe (X10 was invented in Scotland) that means that each command takes 1 second. For those of us on 60 cycle AC the each command takes ~0.833 seconds. Here is the break down (this assumes nobody is sending or in the middle of sending): 1) ( 3) The sending device should wait for 3 cyles with no signal seen. 2) ( 2) Then send the Start of Header (1110) this takes 2 cycles. 3) ( 9) Then send the first half of the command (Letter code/Number code) this takes 9 cyles. 4) (11) Repeat steps 2 & 3. 5) ( 3) Pause for 3 cycles. 6) ( 2) Then send the Start of Header (1110) this takes 2 cycles. 7) ( 9) Then send the second half of the command (Letter code/Function code) this takes 9 cyles. 8) (11) Repeat steps 6 & 7. Done! 3 + 2 + 9 + 11 + 3 + 2 + 9 + 11 = 50 One can argue that the first 3 cycle pause can be ignored but I don't recommend it. Also this doesn't take into consideration the Bright/Dim commands or the Extended Data/Command Commands. These commands don't have the middle pause between the Letter/Number packet and the Letter/Function packet. But did you notice that the total is 50 Hz. Yes it takes 1 sec for X10 to send a command to a device. In Canada and the US it takes 833.33 mS. -- Layer 2 the half bit, bits, packets and errors. --- Standard Definitions: Half bit = the signal present on half a cycle of the carrier Should I switch from using the term 'half bit' [SOH] = Start of Header (1110, first 4 half bit pattern) Hc = House Code (8 half bits) Uc = Unit code (10 half bits) Fc = Function Code (10 half bits) (P) = Pause (000000, 6 zero half bits) (P)+ = Pause plus (6 or more zero half bits) mTW523 outputs a at the 7th zero half bits. ZC = Zero Crossing, where the AC voltage = 0V ------ --- ---- --------------- ------------------ [ Hc Uc Fc Notes ] ------ --- ---- --------------- ------------------ [ 0000 ] M | 13 | All Units Off | ] [ 0001 ] E | 5 | All Light On | ] [ 0010 ] C | 3 | On | ] [ 0011 ] K | 11 | Off | ] [ 0100 ] O | 15 | Dim | ] [ 0101 ] G | 7 | Bright | ] [ 0110 ] A | 1 | All Light Off | ] [ 0111 ] I | 9 | Ext Code 1 | (Ext Code) ] [ 1000 ] N | 14 | Hail Request | ] [ 1001 ] F | 6 | Hail ACK | ] [ 1010 ] D | 4 | Ext Code 3 | (Preset Dim (1)) ] [ 1011 ] L | 12 | Unused | ] [ 1100 ] P | 16 | Ext Code 2 | (Preset Dim (2)) ] [ 1101 ] H | 8 | Status On | (Ext Data) ] [ 1110 ] B | 2 | Status Off | ] [ 1111 ] J | 10 | Status Req | ] ------ --- ---- --------------- ------------------ --- Dim/Bright [ C16 C16 COn COn ] or [ C16 COn ] Repeat V [ SOH ] [ HC ] [ UC ] [ 01 ] ... [ SOH ] [ HC ] [ CMD ] [ 10 ] ... [ 1110 ] [ 01 01 10 01 ] [ 10 10 01 01 ] [ 01 ] ... [ 1110 ] [ 01 01 10 01 ] [ 01 01 10 01 ] [ 10 ] ... 000000 [_SOH__] [______C______] [_____16______] [_Nn_] ... [_SOH__] [______C______] [_____On______] [_Fn_] 17 07:20:09 - 11100110100110101001011110011010011010100101000000 17 07:20:09 - (SOH) A.2 (SOH) A.2 17 07:20:09 - 11100110100101011010101110011010010101101010000000 17 07:20:09 - (SOH) A.Off (SOH) A.Off A 00 is an idle condition (nothing is being sent). A 01 is a 0 bit (if the SOH has been sent). A 10 is a 1 bit And a 11 is can error or the first part of an SOH. --- Extended As far as we know (at the time of this writing) only "Extended Code 1" has a defined frame length which is 31 cycles (62 bits) and is described as: * Start Code = 4 bits, * Housecode = 4 bits, * Extended code 1 = 5 bits (01111), * Unit code (device code) = 4 bits, * Data = 8 bits, * Command = 8 bits.. # Extended code A[1] # #16 00:52:50 - 1110 01101001 0110101010 01101001 10010110.10010110 10011010.01010101 1110 0110100101101010100110100110010110100101101001101001010101000000 # [ SOH ] [ HC ] [ FC ] [ UC ] [ Data ] [ Cmd ] # 0110 01111 0110 1001.1001 1011.0000 # A Ext 1 1 9 9 B 0 #16 00:52:50 - 1110011010010110101010011010011001011010010110100110100101010111100110100101101010100110100110010110100101101001101001010101000000 The explanation for not having a defined frame length for the other two is: "Extended code 2 is variable in length, depending on the type of message. It has its own separate "attention" marker to separate it from all other formats. Extended code 3 has been "assigned" for security but doesn't actually exist yet so its format has not yet been defined." - My intention (theory) My intention for the mTW523 were really simple. I just wanted to see every half bit that was on the AC. The theory is simple enough, block the 60 Hz signal while allowing a 120K Hz signal through. Then convert that 120K Hz signal to a logic 1. The TW523 already does most of this for you except it blocks half the signal. So I decided to modifiy the TW523. I got the idea that if I used a transistor as a switch I could have it on when a signal was present. So I started from there. - Hardware -- TW523 --- Default Operation The TW523 provides a low impedance path for the 120KHz signal and a high impedance path to the 60Hz via a capacitor (C10). It also limits the available current via a 100K resistor (R11). It then proceeds through a isolation transformer and gets clipped by diodes (D3 & D4). It then monitors the half bits for the pattern 1110 which is the Start of header (SOH). The next 8 half bits are the house code. Followed by 10 bits for the unit code or the function code (the last 2 bits determine whether it's a function code (a command) or a unit code). An unmodified TW523 can not receive extended codes The general principle of the the TW523 is to read the 120K Hz signals and filter only valid X10 packages. It reads in 22 cycles of data and passes the last 11 cycles of data to the end device. The chip that X10 uses has been described as an analog shift register I was a little surprised when I came up with the idea to replace the X10 IC with a transistor amp design. I wondered, why had no one else attempted it? The theory is simple enough, a valid X10 signal would have a 120K Hz for 1.04 mS starting within 50 uS of the zero crossing. Since I was lazy I realized that a pulse of a certain voltage level amplified to greater than 5v would look like a TTL logic 1 after it had passed through the opto isolator. I'd see every bit including invalid packets (from noise or collsions). Of course it's not perfect as signals less than nV (1.2V?) pk-pk would not be seen. --- Modified Operation - Software -- PIC After I got the modified TW523's ouput real nice and pretty (well, close enough for government work ....) I needed a way to convert the output to something a computer can deal with. Since I didn't want to bog down the computer with thet details of bits and zero corssings I went with a PIC chip. The reason I chose the PIC's was that I had the Microchip ICD and I had a C compiler for the 14 bit flash chips. I'm only just getting started with the AVR's so they wouldn't have made a good choice in this case. The software in the PIC was much simplier than I thought it would be once a few decisions were made. The first decision was that the PIC wouldn't do the translation. The second was that any 1 half bit would be sent as an ASCII 1 (0x31) and a 0 half bit an ASCII 0 (0x30). A 1 half bit would start the sniffer transmitting and the 7th 0 half bit would cause the sniffer to tranmit a through the RS232 port. It then stop sending data until the next 1 half bit woke it up. This also simplified the code as I could now use the RB0 interrupt to wake the PIC from it's dormancy.
Links:I've also added the ability to write bits to the AC line. Using the Programs on the web site, one can send either fix commands or raw bit string (sent in hex value). I chose to move the encoding/decoding out to the user program to make things simplier within the PIC. The PIC simply receives the bytes and sends out each bit at the zero crossing. This permits not only valid signals but signals that the standard X10 is not able to interpret, such as your own custom protocol. It also allows for us to see what happens on bit errors and packet corruption. Future releases of could could also monitor the AC for the identical signal on other phases (such as all 3 phases as recommended in the X10 Tech. Note). I always wondered about that, here in the states the we use 2 phases in residential homes, the primary phase and an inverted phase. [ need picture ] # -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- # # Data to send # D = 13 bytes # 1 2 3 4 5 6 7 8 9 A B C D # C16Con | | | | | | | | | | | | | | # 14 09:37:21 - <1110 0101> <1001 1010> <0101 0111> <1001 0110> <0110 1001> <0101 0000> <0011 1001> <0110 0101> <0110 0110> <1110 0101> <1001 0101> <1001 1000> <0000 0000> # E 5 9 A 5 7 9 6 6 9 5 0 3 9 6 5 6 6 E 5 9 5 9 8 0 0 # <1110 0101> <1001 1010> <0101 0111> <1001 0110> <0110 1001> <0101 0000> <0011 1001> <0110 0101> <0110 0110> <1110 0101> <1001 0101> <1001 1000> <0000 0000> # [SOH] [ 01011001] [ 1010010101 ][ SOH ][ 01010110 ][ 1010010101 ] [ Pause ][SOH][ 01011001 ][ 0101100110 ] [SOH][01011001 ] [0101100110 ][ Pause ] [ Extra ] # [ 0010 ] [ 11000 ] [ 0001 ][ 11000 ] [ 0010 ][ 00101 ] [ 0010 ] [ 00101 ] # [ C ] [ 16 ] [ C ][ 16 ] [ C ][ On ] [ C ] [ On ] # -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Other options would be to add an ADC circuit (many of the PIC now include an ADC). We could then measure the voltage levels of the signal. Sending it through the RS232 would be simple as a 0 would still mean a 0 half bit. But 1 - 255 would indicate a signal level (divided by 255) as the 1 half bit. -- PC I wrote the first version of the PC's app in Perl. Originally It was written on a Linux box and I used the Curses module and the Device Serial module. I hope to rewrite it so that it doesn't use the Curses module. This module may not be available on Windows machines and I'm sure that more than a few of you are not ready to load one of the free Unixes. Now, the first thing we do is to setup the variables that we need then we setup the serial port (9600, 8, N. 1, no handshaking). In the curses version we setup our windows and open our log file. In the while(1) loop we get down to work. Next we setup select and for many of the non Unix programmers you've probably not seen select(). Select lets us sit there and wait for something to happen with out polling the items of interest. This reduces the CPU utilization dramatically. When select returns we most likely have something to do. Next we check for what that may be. If it's input from the serial port we take it and parse it. So far my translator assumes a perfect string. From what I've seen I really need to fix this as my X10 exists in a dirty environment. I'll fix that later. After we parse the line we print it to the appropriate window and the log file. The other item of concern is the user input. Acceptable inputs are quit, clear a window or both, and adding a comment to the log. In the original version of the PIC code I'd send a line of data 1 character at a time and terminate it with a
- PIC to mTW523 code, version A
- PIC to mTW523 code, version B
- PIC to PC Perl code, version B
- PIC to PC Perl code, version II
. This worked great and I'd see all the bits and not worry about the 'dead air' between commands. But I started to notice that some commands that should be grouped together were being sent as 2 or more parts. This meant that there were at least 7 zero crossing with zero half bits. I couldn't tell how many bits were being sent so I'm now rewritting it to send all the bits including the idle bits. This means a lot of the logic for determining an X10 packet is being moved out to the users application. - The sending of data Now being able to see all the bits on the AC is great but I also wanted to know what happens when a packet gets sent. Especially when that packet has corruption in the first or second half of the packet. So I decided that the method for sending X10 would be handled at a byte level. That is to say that the packets would be made of bytes sent from the users app and the PIC program would shift each bit on the half cycle out onto the AC. This would allow us to send good and bad packets or even create our own formats. From the users app we would decode things like A1 on to the appropriate bits but also permit the sending of manual packets. Flow control was a bit of a problem which puzzled me for a while. I didn't want to add more logic to use hardware flow control. This would allow me to possibly use the smaller 12Cxxx series of PIC's. The reduction in parts would simplify the building of the device. I also couldn't use software flow control as if I ever decided to send the analog level of the X10 signal it could be any value between 0 and 255. Instead I opted for the users app to monitor the X10 bits and determine how many bytes had been sent. Links:
You may email me at ncherry@linuxha.com.