* Executive Summary This folder contains files related to the development of a Farm "Network" that allows communications between devices, and to some degree, with animals. The devices are typically linked via the Internet/Ethernet. At the moment, there will be: * XBee based "Satellite" devices, that are for placing on animals/portable devices and may collect various sensor information and transmit it to "Anchor" devices. * "Anchor" devices, that typically have GPS, WiFi, power, and maybe server capabilities. The typical set up will be 3 Anchors spread within one mile of each other, and a potentially large (up to 1000) Satellite devices. Louis Thiery is suggested using the XBee's. Bruce Dawson is working on the software. Miles Smith Farm and a SARE grant is providing some funding. The project is being collorated with on www.farmhack.net. This document is heirarchialy organized; each '*' at the first of a line begins a section. A single '*' is a first tier "section" (think "1.0"); two asterisks ("**") indicates a second tier section (think "1.1"); three asterisks ("***") indicates a third tier section (think "1.1.1")... * Block Diagrams ** Anchor CPU <-> GPS (base location and timing) CPU <-> XBee (satellite/ear tag location) CPU <-> Ethernet (upload data to display server) ** Satellites XBee (to echo location information back to Anchor) ** Display Server The display server performs most of the computation required to determine the absolute position of each satellite. This is because the location information is derived from data indicating different points in time and different sampling location; a statistical database is required to correlate all the various and related data. CPU <- GPS data CPU <- Satellite timing info CPU -> Absolute position at a given point in time * Flow of Operation ** Anchor Most of the logic is in the anchor. It initiates the distance checking with the satellites, stores the data in a database, and then uploads it on request to a server. *** Power Up Upon power up, each Anchor/Base station will perform the following tasks. Error/warnings will be indicated in a log file, and a red light on the CPU will flash. **** Ensure GPS and XBee is attached. Initial version numbers and other data will be recorded in the logs along with operating state. ***** XBee This section covers the XBee's technical details; including communication protocols and commands. When an XBee is connected to a Ubuntu 14.03 LTS box (the test platform), the follow will appear in /var/log/syslog: Jul 19 10:46:17 db kernel: [153266.958408] usb 5-2: new full-speed USB device number 2 using uhci_hcd Jul 19 10:46:17 db kernel: [153267.156331] usb 5-2: New USB device found, idVendor=0403, idProduct=6001 Jul 19 10:46:17 db kernel: [153267.156337] usb 5-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3 Jul 19 10:46:17 db kernel: [153267.156341] usb 5-2: Product: TTL232R-3V3 Jul 19 10:46:17 db kernel: [153267.156343] usb 5-2: Manufacturer: FTDI Jul 19 10:46:17 db kernel: [153267.156346] usb 5-2: SerialNumber: FTG90KYF Jul 19 10:46:17 db mtp-probe: checking bus 5, device 2: "/sys/devices/pci0000:00/0000:00:1a.2/usb5/5-2" Jul 19 10:46:17 db mtp-probe: bus: 5, device: 2 was not an MTP device Jul 19 10:46:18 db kernel: [153267.956794] usbcore: registered new interface driver usbserial Jul 19 10:46:18 db kernel: [153267.956809] usbcore: registered new interface driver usbserial_generic Jul 19 10:46:18 db kernel: [153267.956818] usbserial: USB Serial support registered for generic Jul 19 10:46:18 db kernel: [153267.968543] usbcore: registered new interface driver ftdi_sio Jul 19 10:46:18 db kernel: [153267.968557] usbserial: USB Serial support registered for FTDI USB Serial Device Jul 19 10:46:18 db kernel: [153267.968659] ftdi_sio 5-2:1.0: FTDI USB Serial Device converter detected Jul 19 10:46:18 db kernel: [153267.968687] usb 5-2: Detected FT232RL Jul 19 10:46:18 db kernel: [153267.968688] usb 5-2: Number of endpoints 2 Jul 19 10:46:18 db kernel: [153267.968690] usb 5-2: Endpoint 1 MaxPacketSize 64 Jul 19 10:46:18 db kernel: [153267.968692] usb 5-2: Endpoint 2 MaxPacketSize 64 Jul 19 10:46:18 db kernel: [153267.968693] usb 5-2: Setting MaxPacketSize 64 Jul 19 10:46:18 db kernel: [153267.971436] usb 5-2: FTDI USB Serial Device converter now attached to ttyUSB0 ****** AT Command set The following AT commands will be sent to Anchor point XBee on start up: VL - Get version of firmware, hardware, ... Returns something like: XBee 802.15.4 BETA V 10EC, Build: Feb 22 2011 17:05:54 Hardware Version: W45? MC13213 Software Compatibility: 01 XCVR_ID: 682F MAC FFDNB V1061 Build: Jul 23 2007 14:47:59 PHY XBEE4 V1061 Build: Feb 22 2011 16:41:52 MAX BOOTLOADER V 0B - or - XBee-PRO 802.15.4 BETA V 10EC, Build: Feb 22 2011 17:05:54 Hardware Version: X47 MC13213 Software Compatibility: 01 XCVR_ID: 6800 MAC FFDNB V1061 Build: Jul 23 2007 14:47:59 PHY XBEE4 V1061 Build: Feb 22 2011 16:41:52 MAX BOOTLOADER V 0B EC - Get Clear Channel Assesment value - if not zero, then we need to use a different channel, but all anchor points have to be set to the same channel, so this is an error. EA - Get ACK failues; if non-zero on power-up, then faulty XBee hardware is probable cause. Note that this tends to increase around other wireless devices. ED1 - Get energy on each channel. This information is stored for diagnostic purposes. ND - Returns the nodes that are within range of the Anchor Point. These are recorded and new node entries are initialized. NI - Any existing node name is logged, but then replaced with the one set in the configuration file. DD - Returns 10000 indicating the type of XBee product. SH - Serial Number/High. Returns 13A200. SL - Serial Number/Low. Returns 40B7C07C. CH - RF Channel. Should return 'C' for XBee-PRO, 'B' for XBee. CE - Routing mode (0=router, 2=end device) AP - API mode (0=straight modem, 1=On w/o escapes, 2=On w/escapes) AO - Error BD - Baud rate (0=1200, 1=2400, 2=4800, 3=9600, 4=19200, 5=38400, 6=57600, 7=115200, 8=230400) RO - Packetization Timeout (0=no buffering). Returns 3 FT - FLow control threshold. ERROR NB - Parity (0=None, 1=Even, 2=Odd, 3=Forced high, 4=Forced low) D7 - DIO7 configuration (0=Unmonitored Input, 1=CTS, 3=Monitored Input, 4=Low output, 5=High output, 6=RS-485 low TX, 7=RS-485 high TX). D6 - DIO6 configuration (0=Unmonitored input, 1=RTS, 3=Monitored input, 4=Low output, 5=High output) VR - Firmware version. Returns 10EC. HV - Hardware version. Returns 1847. CK - Configuration code for current AT command set. Returns D8. ER - RF errors. Error. GD - Good packets. Error. RP - RSSI PWM timer in tenths of a second. (Signal strength). TR - Transmission errors. TP - Temperature. Error. DB - Received signal strength measured in -dBm. (A return of 60 means -95dBm.) CT - Command mode timeout. CN - Exit command mode. GT - Guard time. CC - Command Character. Hex value of character. ID - Network ID. Returns 3332 NT - Node discover timeout. NI - Node identifier. Ends with ',' or CR. Max 20 bytes. We're using BASEx and TAGxxx. DN - Discover Node/Destination Node. Resolves a NI to a physical address. Sets DL & DH. Returns OK. Exits command mode. ND - Network discover. Following returned for each module: MY, SH, SL, NI, PARENT_NETWORK_ADDRESS, DEVICE_TYPE, STATUS, PROFILE_ID, MANUFACTURER_ID, . NO - Network discover options. (0x01 = append DD value. 0x02 = local device responds too.) MT - Broadcast Multi-transmit - all packets are transmitted MT+1 times. RR - Unicast MAC retries. Returns ERROR. PL - RF Transmission power level. XBee-PRO is 0=10dBm, 1=12dBm, 2=14dBm, 3=16dBm, 4=18dBm. XBee is 0=-7dBm, 1=-1.7dBm, 2=-0.77dBm, 3=0.62dBm, 4=1.4dBm. SM - Sleep mode (0=None, 1=Pin sleep, 4=Asynchronous cyclic, 5=Asynchronous cyclic w/pin wakeup, 7=sleep support mode, 8=synchronous sleep mode. SO - Sleep options. Bit mask. bits(0=Preferred sleep coordinator, 1=non-sleep coordinator, 2=send API sleep status messages, 3=Disable early wake up, 4=Enable node type equality. 5=disable lone coordinator sync repeat, 8=Always wake for ST time. ST - Wake time. The amount of time to stay awake after recieving RF or serial data. SP - Sleep time. MS - Missed syncs. SN - Number of sleep periods. SQ - Missed sync count. SS - sleep status. Bitmask. (0=network is in wake state. 1=node is network sleep coordinator. 2=received sync since power. 3=received sync in current wake cycle. 4=will become sleep coordinator and sync on next wake. 5=commissioning requested sleep coordinator. 6=currently in deployment mode. OS - Operational sleep period in tens of milliseconds. OW - Operational wake period in milliseconds. WH - Wake host. ****** API protocol The API mode is turned on with ATAP=1 or ATAP=2. 1 is straight API mode with no escape characters. 2 is API mode with escape characters. In both cases, the data frame sent to the UART has the following blocks: 1. 0x7E start of transmission 2. Length (2 bytes) MSB sent first, followed by LSB. 3. Frame data. 4. Checksum Data sent before the 0x7E is discarded. The frame is discarded if the checksum fails or there are transmission errors. Escaping is accomplished by sending 0x7D followed by (the byte to be escaped XOR'ed with 0x20). The following bytes need escaping: * 0x7E - Frame Delimiter * 0x7D - Escape * 0x11 - Xon * 0x13 - Xoff The checksum is calculated from the Frame Data by adding all bytes, keeping only the lowest 8 bits of the result and subtracting it from 0xFF. **** Configuration File There is a configuration file stored on each Anchor point that has the data particular to that Anchor point. ID: A 20 character (or less) string indicating the name of the anchor point. This can not be the same as any other node (anchor point or satellite) in the same field as this anchor point. *** Standard Operating State After the power-up sequence has been performed, then several other operations are performed. **** Periodic Operations These are typically cron jobs that occur every X minutes: Note that at this point of development, absolute location determination is a two step process involving data from both the GPS' on the Anchor point and the distance computation through the XBee's. Given the anemic computational power of the Beaglebone, and the temporal nature of the distance information, it makes sense to keep the GPS and XBee distance information in separate databases, and let the absolute positioning computation to be done in the display server. This will allow one to re-compute the same distance at a later point in time even though an anchor point may have changed position since the original data was taken. So, in a nutshell, absolute positioning (using lattidue and longitute) is not available in the anchor points. The anchor points only have relative position to each satellite. The data from each anchor point has to be "merged" and then correlated and computed to determine the absolute position of each satellite at a particular point in time. ***** Satellite Location Sensing The XBee ND command with be used to find all satellite XBee's within range. Then each XBee found will be polled and timed to determine distance. This distance information will be recorded in a CSV file that can be subsequently uploaded via the ethernet cable. At some point in the near future, this information will be consolidated/replicated with all the Anchors so there will be either: * A master with all the information from each Anchor point * All Anchor points will have the same info as all the other Anchor Points. The XBee command "ND" returns a list of nodes with the following annotations: MYSHSLDBNI The end of list is indicated by three consecutive 's. MY: The sending Anchor point's 16 bit address; SH, SL: the satellite's 64 bit address as two 32 bit address - high and low order digits; DB: Signal strength; NI: Node name ***** Anchor Location Sensing Anytime a change in GPS coordinates for the Anchor point is detected, the new location is "statistically averaged" with prior locations and the "new location" is updated. This is used in determining the absolute location of each ear tag. It is expected that the Anchor Location will take a few days of samples to "settle". However, sampling will continue so that it can determine if its physical location really does change (if a limb knocks it down, or a farmer moves it). Note that an alternative (but less accurate) method of distance measuring is by using the XBee's DB command - which returns the signal strength in -dBm (values are between 0x17 and 0x64) - of the last good packet received (from whatever transmitted it). ***** Anchor Health The following commands are executed every so often to keep track of the Anchor point's health and to indicate general trends in possible failures: EC - Get Clear Channel Assessment values EA - Get ACK failures If either of the above return non-zero, then the event is recorded and the value is reset to zero. ***** Anchor Data Transmission The following data is transmissed from each anchor to the "central server". This information will be used (at the off-site central server) to determine the coordinates of each Satellite at given times. ****** Anchor Location This data stream indicates where the anchor is and its current health. Also, the "deviance" between readings will be sent to correct any change in time. 01 - Record Identifier - when the data was taken - the station reporting the info - the location info from the GPS - the location info from the GPS - the RTC's time deviation from the GPS's "beat per second". ****** Satellite Location This data stream indicates where each sateliite is. 02 - Record Identifier - when the data was taken - the anchor/Base Station that recorded and is reporting the info - the satellite/ear tag that the following data was recorded for. - the delay between the station and the satellite. The delay data is used to indicate distance from the base station/anchor. ** Satellites The satellites (aka "ear tag") are very simple systems that only echo received data back. In some future version, they may be able to support sensors and actuators. For now, only thing they do is echo data back from the Anchors. * Provisioning This section describes what has to happen when new hardware is received from the manufacturer; it has not yet been configured for use in this application. This section describes the configuration details. ** Beaglebone Black First, connect a USB-A cable between the Beaglebone Black (bbb)'s client USB port (a Mini-USB connector), and a host computer. Preferrably, the host is running Ubuntu Linux rev 12.04 or later. Do Not - at this time, connect any other peripherals to the BeagleBone Black - the computer won't power up if you do. You can connect the peripherals *after* the network is set up (See "Set up the network" below.) The host computer should recognize the bbb as having both an ethernet and terminal access over the USB. The 192.168.7.1 network interface will be automatically configured on the host computer (typically as eth1 or eth2); the bbb will have the IP address 192.168.7.2. Use ssh to connect to the bbb from the host. ssh debian@192.168.7.2 Note that only the host can connect to the bbb. Other systems can not connect to the bbb unless they come through the host computer. The password is 'temppwd' (without the quotes). There can be no capital letters. You will change this password within the next few steps. *** Set up the network The next step is to set up the network so you can copy files to it and use it interactively. Note that you will have to set up your DHCP server first. **** /etc/network/interfaces Uncomment the following lines from this file. You will have to edit this file as the superuser, so use something like 'sudo vi /etc/network/interfaces'. #allow-hotplug eth0 #iface eth0 inet dhcp Once uncommented, the lines should look like: allow-hotplug eth0 iface eth0 inet dhcp Make sure you uncomment the lines with "eth0" in them, not "eth1"! If you uncomment the wrong lines, then you will not be able to access the network. This will enable the network interface. **** /etc/hostname This file should have just one line: the name of the system (beaglebone). Edit the file to have the hostname you want (bbb). This will have to be done as the superuser. After editing the above file, you will have to run the following to change the hostname until you reboot. $ sudo hostname bbb **** /home/debian/.ssh/authorized_keys This file determines who to allow to connect to this account using SSH. The content of this file comes from your host computer. Use the following command on the host to create the file; this must be executed on the host (not the bbb). So log out from the bbb so you will be on your host and enter the following commands. $ ssh debian@192.168.7.2 "mkdir -p .ssh" $ ssh debian@192.168.7.2 "cat >>.ssh/authorized_keys" < ~/.ssh/id_dsa.pub In the above, you will probably have to enter the password for the debian account. It defaults to temppwd. Now test that the above command had the desired effect by logging in to the bbbb. *** Change the password for the debian account The default account on the bbb allows superuser access. So you need to change its password. $ passwd (current) UNIX password: temppwd Enter new UNIX password: Retype new UNIX password: *** Reboot Now, reboot the bbb with the following command: $ sudo shutdown -r now This reboot will enable the network. While it is rebooting, you should attach the network cable; the bbb will now have access to the network and you can download software to it. *** Set the local timezone Run the following command and answer the prompts to set the correct time zone: $ sudo dpkg-reconfigure tzdata If the time is still incorrect after running this package, then run: $ sudo ntpdate 0.debian.pool.ntp.org *** Install additional packages The following additional packages were installed on the BBB: * minicom - provides terminal I/O (human interaction) to the devices. * sqlite3 - provides database software used by the GPS. * libsqlite3-dev (only needed if you will be changing the code) * wireless-tools iw - provides Internet access over WiFi. The above packages are available through the usual Debian repositories using 'sudo apt-get update', and 'sudo apt-get install ...'; like so: $ sudo apt-get update $ sudo apt-get install minicom sqlite3 libsqlite3-dev You can ignore error messages about "W: There is no public key available for the following key IDs: XXXXXX" Otherwise answer "Y" to the following prompt: After this operation, 2,205 kB of additional disk space will be used. Do you want to continue [Y/n]? *** Install the "kit" The "kit" contains all the files necessary for this project to work. It must be installed on each BeagleBone Black Base Station/Anchor point for this project to work. After installing the files in the kit, they must all be configured to work on your base station(s), which means you'll have to build all the software. Included with the kit is LibXbee - a set of routines used to communicate with the XBees at the API level. This library is used extensively in this software, so it must be built. To install the kit, you must first copy the kit file to the bbb from your host: $ scp SARE-armv7l-3.8.13-bone47-kit.tgz bbb: (The above assumes that the Beaglebone Black you are copying the kit to has a network name of 'bbb'.) Then, log in to the bbb as user 'debian', and use the following commands: $ cd / $ sudo tar xzf ~debian/SARE-armv7l-3.8.13-bone47-kit.tgz This will update the /home/debian directory with the necessary files. And put one file in /usr/local/bin/msf. The programs in the /home/debian/bin directory are: gpscap - captures information from the GPS unit. pingtags - Measures the time required to "ping" each eartag. xbeeat - a diagnostic program for the XBee hardware. scantags - Lists the tags the local XBee can see. settagid - Changes the parameters on a tag. The /usr/local/bin/msf is a program script that should be run when the system is booted. It creates the UART multiplexed devices that the GPS and XBee are connected to. It must be run using: sudo /usr/local/bin/msf It does not produce any output. If an error message was produced, then please contact the developer. Note that there will be a directory tree /home/debian/MSF/Projects/FarmHacks/Network, which will contain all the source files required to reproduce this project. *** Install the XBee library The programs use a library to communicate with the XBee devices. You will have to "install" this library by executing the following commands: $ cd ~debian/MSF/Projects/FarmHacks/Network/LibXBee $ sudo make clean install This will output a number of lines (beginning with "install...", "chown ..." or "ln -...") indicating its progress. These can be safely ignored. The development environment needs a "library" directory to store system-specific libraries: $ mkdir ~debian/lib *** Create the GPS database The database used by the GPS system (gpsinfo.sqlite3) needs to be created before its used. The 'makedb' program script does this like so: $ makedb Note that this only has to be done once - typically when the software is first installed, or the base station has been moved. If it is re-run, then errors will be produced indicating the tables already exist. *** Update the tags file The pingtags program uses a 'tags file', which has all of the XBee transceiver IDs that the software periodically checks. (The IDs of each of the "ear tags" that are worn by the animals). Note that the 'tags file' can be of any name and anywhere on the bbb, just so long as the scantags program can read it. For example: ~/cattle.tags Each ear tag ID must be one per line, in hexadecimal notation. For example: # This is a test .tags file that lists the names of all the tags we're tracking. # These names are the same names that are stored in the XBee hardware as # the "Node Identifier". 0x1001 0x1002 # another comment 0x1003 #comment The ID numbers are assigned by you; one per tag and are put in this file using a text editor such as nano, vi, emacs, ... (NOT a Word Processor). The scantags program will list all tags that have the same Network ID (PAN ID), and Channel that are within range of the base station. For example: scantags -d /dev/ttyO2 -a -v -l (NB: the scantags program won't work until the peripherals have been connected.) The settagid program can be used to initially set the parameters on each XBee. Note that once the Channel and/or PAN have changed on an XBEE, it can't communicate with others unless they have the same Channel and PAN. *** Connect the peripherals Connect the peripherals as described in the "Hardware Connections" section below. You should shutdown the bbb before you connect the peripherals. Use this command: $ sudo shutdown -h now When all the lights stop, unplug the power and perform the connections as described in the "Hardware Connections". Once you have verified the connections, restart the bbb by plugging in the USB/power. Lights on the peripherals should begin to flash, if they don't please contact the developer after you have verified the connections (the XBee connections are particularly hard to verify as the board pin-outs are on the bottom of the board!). You can now SSH into it and run the command $ sudo /usr/local/bin/msf This command has to be run everytime the Beaglebone Black is rebooted or powered on. If it is not run, the utility programs will not work properly. *** Connect to the server In order to prevent errors from SSH that prevents sending data to the server, you will have to connect and get the host keys set up between each BBB and the server. $ sudo ssh -i /home/debian/.ssh/id_rsa codemeta.com@simon.codemeta.com id This will issue a prompt about being sure that you're connecting to the host; just respond with "Yes" since this is the first time you're connecting to the server. Then it will print out a line similar to: uid=1002(codemeta.com) gid=1002(codemeta.com) groups=1002(codemeta.com),4(adm),27(sudo) NOTE TO READER: The text of this message will probably change when this goes to "production" because I don't want people getting into my server. (Or I'll just pre-populate the known_hosts file.) *** Start the utilities. Now, the utilities have to be started. These should run all the time. Please start these utilities in the below order (GPS first, then XBee PRO S1). **** GPS Start the GPS monitoring utility - gpscap - using the following command: $ gpscap /dev/ttyO1 & **** XBee PRO S1 Ensure that the XBee is set to API mode (instead of Transparent mode). Use the command: $ setapi -d /dev/ttyO2 -o 1 The above will print out a few lines indicating that it has (or has not) sent the command to turn on API mode. If there's an error, please contact the developer. Next, Start the 'pingtags' program using the command: $ pingtags -d /dev/ttyO2 -t cattle.tags -l cattle.log & Note that 'cattle.tags' should be replaced with the tags file you created in "Update the tags file" section for the other bbb's in your application. To update a remote database, I use the following command: $ pingtags -d /dev/ttyO2 -t test.tags -l "|bin/senddata.sh" (If the argument to the -l option starts with a "|" (pipe), then the logging output is sent to the stdin of the program named immediately after the pipe character. Typically, you will have to put this argument in quotes.) * Operating After provisioning is complete comes the daily hum-drum of operating. This section covers the anticipated cases of daily operations. In general, the system will continue to operate by itself indefintely. However, there will be times where the system, or part of it, has to be restart for mundane things like changing batteries, or shutting down because there are no animals, ... The subsections below describe some anticipated "use cases" that the typical user will encounter. ** Power cycling base stations ** Replacing batteries in ear tags ** Moving to a new pasture/location * Hardware Connections The following describes the hardware connections between the Beaglebone Black and the various devices. ** Power 5 Volt DC - using a wall brick. Alternatively, the USB client port can be used. ** GPS 3.3V - P9/3 - Provides power to the GPS GND - P9/1 - Provides ground (power) to the GPS TX - P9/24 - The GPS's Transmit channel (This is where data is output). RX - P9/26 - The GPS's Receive channel. EN - P9/19 - The GPS's Enable circuit - useful for connecting to CTS on a UART to prevent data overruns. The GPS with the above pin assignments is access through /dev/ttyO1 - note: That's Oh-one The default speed on the GPS appears to be 9600N8. When testing inside, you will want a remote antenna connected to the GPS, otherwise you won't get a fix. ** XBee With the XBee adapter inserted in the socket, and the XBee Pro mounted on top of the adapter, the pins are LH (Left hand side), and RH (right hand side). These are marked on the underside of the adapter card that the XBee plugs into. Be careful to not mix up the pins when you flip the card over. LH VCC - P9/3 - Provides 3.3V power to the XBee LH DOUT - P9/22 - Data out from XBee LH DIN - P9/21 - Data in to the XBee LH VSS - P9/1 - Signal Ground RH RTS - P8/37 - RTS/CTS (Not verified) RH CTS - P8/38 - RTS/CTS (Not verified) The XBee is accessed with the above pin assignments through /dev/ttyO2 - note: That's Oh-two. The default speed on the XBee appears to be 9600N8. * Notes/Journal/Log ** 23 Feb 2016 Started adding the Real Time logic to pingtags to try and get the deltatimes down to a more reasonable number. Not sure if it will work as Linux only does soft Real Time, but we'll see. ** 15 Feb 2016 Been out sick for almost a week now. Not much progress. Didn't even get the big batteries recharged for the base stations. This past Saturday's demo/test with Dot was moved to 16 Feb 2016 because she's also sick. The only thing I have ready for the test/demo are the halters are set up with tags (and the "protective pouches" for the tags). Otherwise, I have to move the base stations around (after the batteries are charged - I forgot that our big charger died this past summer, and never replaced it). And I have to set up a laptop to look at the data. Ah crud. Carole just changed things - we have to use the Cemetery field for testing; well, I'll just reposition the Nano in the barn and put base stations in the cemetery itself, the crotch of the Y in the road, and down near the hay bales at the far end of the road. ** 9 Feb 2016 Made more ear tags today, in spite of the cover and power problems. At least as we progress, we can use these as "building blocks". The USB board needs 5.5V (from the USB), so I'm connecting the 3.3V battery cartridges directly to the XBee pins. I'm not sure what will appear on the USB connector, but there shouldn't be a battery in there when a USB cable is plugged in - something might blow. In order to get 5 tags, I'll have to use the two lower powered XBee's, but I just tried one, and it can't be seen by any of the outside base stations. I'll have to run some distance tests to see if it really can be seen! ** 8 Feb 2016 I found the XBee USB mounting pads! All 6 of them! Note that the XBee's tapered end has to go on the end with the pins (away from the USB connector - otherwise the XBee's shielding shorts out the pins.) These are good - now I don't have to cut the tags - just hot glue on these mounting pads, and I can solder the wires onto the mounting pad. This makes configuring the XBee's a *lot* easier! (Along with diagnosing any problems.) Erk. Evidently, pingtags didn't restart when I restarted bbb1 after replacing the battery. Sigh. I just restarted it on bbb1. ** 6 Feb 2016 I ran over bbb2's battery with the Bobcat this morning, hence no data since then. And the data up until 15:03 today will be suspect because pingtags was running, causing congestion. But as of 15:05, data should be OK. I also moved bbb2 further away from bbb1 by 10 feet (so bbb2 should now be 40' away from bbb1). At 21:43, I took another reading with a 40' separation. The average is 51089621.9337 and the min was 50365459 and the max was 56121584. The difference is 25597. ** 3 Feb 2016 bbb1 ran out of battery last night (it was one of the small trailer/UPS batteries), so I replaced it with a recently charged small automotive battery. Also, bbb1 is now the only one running pingtags, and its pinging ONLY bbb2 (I even killed the pingtags on bbb3 and removed all the other tags in the cattle.tags file). This was just before 10am. Now looking at qtiplot so I can visualize the data I've got. Well, looks like gnumeric has the better plotting abilities. And I'm using mysql-navigator to create the datasets - because both qtiplot and gnumeric seems to be able to graph only two columns from .csv files (which are what's produced by mysql-navigator). (mysql only writes to the server - so I had to set up an ssh VPN to talk to the server on simon... ssh -p 8022 -f -N -L 13306:localhost:3306 simon.codemeta.com & Its looking like the average deltatime for 10' is 51007620.1272 with a minimum of 50206458 and a maximum of 51612834 between 11am and noon. For noon to 1pm, the average is 51013569.0629 with a min of 50276625 and a max of 51959125. Now, let's do a 20' separation... Move to 20' separation was completed at 13:39, so we'll check again at 14:40. And the numbers for the 14:40 reading are an average of 51055605.2771 and a min of 50322042 and a max of 59158042. This is good... we have a slightly larger averages and mins. Given the above, its seems to be about a 42,036 nanoseconds per 10'. But we need to confirm that with many more measurements. THEN we get to throw in congestion! I moved bbb2 to about 30' away around 17:30. The average for between 17:30 and 18:30 is 51072287.1457, min is 50392667, and max is 56718500 nanoseconds. However, excluding all points above 52000000 (14), the average, minimum, and maximum are: 51035025.7284, 50392667, 51943708. (This was produced with "select min(deltatime), avg(deltatime),max(deltatime) from basedata where stamp > 20160203173000 and stamp < 20160203183000 and base = 17 and remote=34 and rssi > 0 and deltatime < 52000000;") The latter number (51035025.7284), indicates the difference from 20' is -20,579 nanoseconds, which is a non-intuitive result. Not excluding any datapoints give us a difference of (51072287 - 51055605 =) 16682. So now we need to try a different sample set, this time from after dark at 19:30 to 20:30... (the sun was setting and directly in the view of the XBees between 17:30 and 18:30). The average, minimum, and maximum are: 51064024.8417, 50200917, and 55961042 - these numbers are without excluding any data points. The difference with the 20' average is (51064025 - 51072287 = ) -8262, which is indicating a trend, but not helping me. So, I need to repeat this in daylight to see if that makes a difference. Also, we should probably check the number of samples. ** 2 Feb 2016 I've been doing some research on an Android app for the past few days, but have run into problems with the server-side - the protocol is not well described. Meanwhile, pingtags on bbb1 has been running non-stop since 21 Jan 2016 18:16:44, which is good. I have to set thing up for outside testing today. So I think I'll put this pingtags on the other base stations now. Note: I'm beginning to believe that its the packet congestion that's causing the XBees to fail in such weird way. But I think the current logic in pingtags will handle it - it'll just exit when things get too bad. Also, I suspect not having the base stations ping each other will help. Also, got bbb1 configured for outdoor use, but I'm not sure its antenna is properly configured. It seems the autoip is different on them. bbb2 and bbb1 have been placed outdoors. There are now 63 records on the server, and the average deltatime is 51143732.7778 nanoseconds. bbb2 and bbb1 are 10 feet apart. The rssi is a consistent 47. Note that bbb1 has a small battery on it while I recharge another automotive battery for it to use. bbb2 is on a medium sized tractor battery... base #34 = bbb2. Remote #17 = bbb1. mysql> select count(*),avg(deltatime) from basedata where stamp > 20160202153100 and base=34 and remote=17; +----------+----------------+ | count(*) | avg(deltatime) | +----------+----------------+ | 63 | 51143732.7778 | +----------+----------------+ 1 row in set (0.05 sec) And a little over an hour later... mysql> select count(*),avg(deltatime) from basedata where stamp > 20160202153100 and base=34 and remote=17; +----------+----------------+ | count(*) | avg(deltatime) | +----------+----------------+ | 873 | 51317129.9107 | +----------+----------------+ 1 row in set (0.04 sec) That's a lot of variance since nothing has moved. I'm now going to turn off bbb2 pinging bbb1 to try and relieve congestion... at 19:16 (almost an hour later): mysql> select count(*),avg(deltatime) from basedata where stamp > 20160202153100 and base=34 and remote=17; +----------+----------------+ | count(*) | avg(deltatime) | +----------+----------------+ | 914 | 51315674.3184 | +----------+----------------+ 1 row in set (0.04 sec) and switching the base and remote: mysql> select count(*),avg(deltatime) from basedata where stamp > 20160202153100 and base=17 and remote=34; +----------+----------------+ | count(*) | avg(deltatime) | +----------+----------------+ | 2719 | 51312330.4160 | +----------+----------------+ 1 row in set (0.05 sec) and within the past hour: mysql> select count(*),avg(deltatime) from basedata where stamp > 20160202183100 and base=17 and remote=34; +----------+----------------+ | count(*) | avg(deltatime) | +----------+----------------+ | 593 | 51294306.4216 | +----------+----------------+ 1 row in set (0.05 sec) And with the extremes: mysql> select count(*),avg(deltatime), min(deltatime), max(deltatime) from basedata where stamp > 20160202183100 and base=17 and remote=34; +----------+----------------+----------------+----------------+ | count(*) | avg(deltatime) | min(deltatime) | max(deltatime) | +----------+----------------+----------------+----------------+ | 608 | 51296208.2632 | 50234083 | 56566917 | +----------+----------------+----------------+----------------+ 1 row in set (0.05 sec) ** 29 Jan 2016 Well, the XBee's have all hung (again). Seems I can't get pingtags to run longer than 24 hours. I went on bbb3 to look at things... Things appear normal from 2016-01-27 22:15:38 up until 20160128143705.9852 (there are errors, but they are successfully ignored). There are Unknown errors in there, so I may have a problem with the code when it outputs error messages, or there may be a bug/problem in LibXBee. But this is the point when there are NO LONGER ANY XBee transactions that aren't in error - all the CH commands fail. And all the error codes are in the range of 231-234, 239, and 301. Note that I changed the code to print the error number using the '%hhd' format string, but this doesn't appear to make much difference (as I anticipated). ** 28 Jan 2016 This is looking good... The basestations have been running with the new pingtags for the past 24 hours, and there have been no interruptions... However TAG001 seems to have lost its name, so I just reset it with XCTU. Here's some stats... mysql> select count(*) from basedata where stamp > 20160128000000; +----------+ | count(*) | +----------+ | 83496 | +----------+ 1 row in set (1.56 sec) mysql> select count(*) from basedata where stamp > 20160128000000 and base=51; +----------+ | count(*) | +----------+ | 28567 | +----------+ 1 row in set (0.01 sec) mysql> select count(*) from basedata where stamp > 20160128000000 and base=34; +----------+ | count(*) | +----------+ | 27848 | +----------+ 1 row in set (0.02 sec) mysql> select count(*) from basedata where stamp > 20160128000000 and base=17; +----------+ | count(*) | +----------+ | 27108 | +----------+ 1 row in set (0.01 sec) mysql> select count(*) from basedata where rssi=0; +----------+ | count(*) | +----------+ | 9124 | +----------+ 1 row in set (0.01 sec) mysql> select count(*) from basedata where rssi=0 and base=17; +----------+ | count(*) | +----------+ | 3288 | +----------+ 1 row in set (0.01 sec) mysql> select count(*) from basedata where rssi=0 and base=34; +----------+ | count(*) | +----------+ | 3480 | +----------+ 1 row in set (0.02 sec) mysql> select count(*) from basedata where rssi=0 and base=51; +----------+ | count(*) | +----------+ | 2363 | +----------+ 1 row in set (0.02 sec) mysql> select * from basedata where base=17 and stamp>20160128162500; +---------+----------------+------+--------+------+---------+-----------+ | entryno | stamp | base | remote | rssi | name | deltatime | +---------+----------------+------+--------+------+---------+-----------+ | 986466 | 20160128162503 | 17 | 4097 | 57 | ? | 50996500 | | 986468 | 20160128162505 | 17 | 34 | 89 | BASE002 | 51554125 | | 986471 | 20160128162506 | 17 | 51 | 61 | BASE003 | 51168958 | | 986475 | 20160128162509 | 17 | 4097 | 57 | ? | 51418750 | | 986477 | 20160128162511 | 17 | 34 | 61 | BASE002 | 50874750 | | 986481 | 20160128162512 | 17 | 51 | 61 | BASE003 | 51044792 | | 986485 | 20160128162516 | 17 | 4097 | 0 | ? | 51496166 | | 986487 | 20160128162517 | 17 | 34 | 92 | BASE002 | 57213500 | | 986490 | 20160128162518 | 17 | 51 | 61 | BASE003 | 51556917 | | 986494 | 20160128162522 | 17 | 4097 | 57 | ? | 50993875 | | 986495 | 20160128162523 | 17 | 34 | 92 | BASE002 | 54576833 | | 986497 | 20160128162524 | 17 | 51 | 61 | BASE003 | 51615791 | | 986502 | 20160128162528 | 17 | 4097 | 57 | ? | 50877834 | | 986504 | 20160128162529 | 17 | 34 | 92 | BASE002 | 54125667 | | 986507 | 20160128162530 | 17 | 51 | 61 | BASE003 | 51592083 | | 986510 | 20160128162533 | 17 | 4097 | 57 | ? | 50982084 | | 986514 | 20160128162536 | 17 | 34 | 0 | | 54060167 | | 986515 | 20160128162537 | 17 | 51 | 61 | BASE003 | 51783542 | | 986520 | 20160128162541 | 17 | 4097 | 57 | ? | 50892583 | | 986523 | 20160128162542 | 17 | 34 | 92 | BASE002 | 51733250 | | 986524 | 20160128162543 | 17 | 51 | 61 | BASE003 | 51669750 | | 986529 | 20160128162547 | 17 | 4097 | 56 | ? | 51066208 | | 986532 | 20160128162548 | 17 | 34 | 92 | BASE002 | 51521583 | | 986533 | 20160128162550 | 17 | 51 | 61 | BASE003 | 51193542 | | 986536 | 20160128162553 | 17 | 4097 | 57 | ? | 51254042 | | 986537 | 20160128162554 | 17 | 34 | 92 | BASE002 | 54270750 | | 986538 | 20160128162556 | 17 | 51 | 61 | BASE003 | 51093416 | | 986545 | 20160128162559 | 17 | 4097 | 57 | ? | 51236166 | | 986550 | 20160128162605 | 17 | 4097 | 57 | ? | 51010000 | | 986552 | 20160128162607 | 17 | 34 | 0 | | 58953125 | | 986557 | 20160128162611 | 17 | 4097 | 57 | ? | 50919417 | +---------+----------------+------+--------+------+---------+-----------+ 31 rows in set (0.02 sec) mysql> select * from basedata where base=34 and stamp>20160128162500; +---------+----------------+------+--------+------+---------+-----------+ | entryno | stamp | base | remote | rssi | name | deltatime | +---------+----------------+------+--------+------+---------+-----------+ | 986479 | 20160128162502 | 34 | 4097 | 64 | ? | 51267500 | | 986482 | 20160128162504 | 34 | 17 | 92 | BASE001 | 51127584 | | 986483 | 20160128162505 | 34 | 51 | 67 | BASE003 | 51537542 | | 986488 | 20160128162509 | 34 | 4097 | 64 | ? | 51165750 | | 986491 | 20160128162510 | 34 | 17 | 95 | BASE001 | 56310375 | | 986492 | 20160128162511 | 34 | 51 | 67 | BASE003 | 51430541 | | 986498 | 20160128162515 | 34 | 4097 | 0 | | 51220875 | | 986500 | 20160128162516 | 34 | 17 | 95 | BASE001 | 56424375 | | 986501 | 20160128162518 | 34 | 51 | 66 | BASE003 | 51269958 | | 986506 | 20160128162521 | 34 | 4097 | 64 | ? | 50972959 | | 986508 | 20160128162522 | 34 | 51 | 66 | BASE003 | 50892542 | | 986512 | 20160128162526 | 34 | 51 | 66 | BASE003 | 71360958 | | 986516 | 20160128162529 | 34 | 4097 | 64 | ? | 51136042 | | 986518 | 20160128162530 | 34 | 17 | 95 | BASE001 | 53619000 | | 986521 | 20160128162532 | 34 | 51 | 66 | BASE003 | 51354917 | | 986526 | 20160128162535 | 34 | 4097 | 62 | ? | 50942708 | | 986528 | 20160128162538 | 34 | 17 | 93 | BASE001 | 51206292 | | 986531 | 20160128162539 | 34 | 51 | 66 | BASE003 | 51362750 | | 986539 | 20160128162542 | 34 | 4097 | 63 | ? | 51017416 | | 986541 | 20160128162548 | 34 | 17 | 95 | BASE001 | 51037167 | | 986543 | 20160128162550 | 34 | 51 | 0 | BASE003 | 51383000 | | 986546 | 20160128162553 | 34 | 4097 | 63 | ? | 50865375 | | 986548 | 20160128162555 | 34 | 51 | 67 | BASE003 | 53720833 | | 986551 | 20160128162558 | 34 | 4097 | 63 | ? | 51066834 | | 986554 | 20160128162559 | 34 | 51 | 66 | BASE003 | 50948167 | | 986558 | 20160128162602 | 34 | 4097 | 0 | ? | 50998125 | | 986560 | 20160128162604 | 34 | 17 | 95 | BASE001 | 53395958 | | 986563 | 20160128162605 | 34 | 51 | 66 | BASE003 | 53821375 | | 986565 | 20160128162608 | 34 | 4097 | 63 | ? | 51118541 | | 986569 | 20160128162610 | 34 | 51 | 66 | BASE003 | 55183375 | | 986571 | 20160128162613 | 34 | 4097 | 64 | ? | 53708375 | | 986574 | 20160128162614 | 34 | 17 | 95 | BASE001 | 59152708 | | 986576 | 20160128162615 | 34 | 51 | 0 | BASE003 | 51142875 | | 986580 | 20160128162619 | 34 | 4097 | 63 | ? | 51010708 | | 986582 | 20160128162620 | 34 | 17 | 95 | BASE001 | 53174458 | | 986585 | 20160128162621 | 34 | 51 | 67 | BASE003 | 51182750 | | 986589 | 20160128162625 | 34 | 4097 | 64 | ? | 51032791 | +---------+----------------+------+--------+------+---------+-----------+ 37 rows in set (0.02 sec) mysql> select * from basedata where base=51 and stamp>20160128162500; +---------+----------------+------+--------+------+---------+-----------+ | entryno | stamp | base | remote | rssi | name | deltatime | +---------+----------------+------+--------+------+---------+-----------+ | 986478 | 20160128162501 | 51 | 17 | 63 | BASE001 | 50646084 | | 986480 | 20160128162502 | 51 | 34 | 67 | BASE002 | 54561500 | | 986484 | 20160128162506 | 51 | 4097 | 71 | ? | 51004917 | | 986486 | 20160128162507 | 51 | 17 | 62 | BASE001 | 50873542 | | 986489 | 20160128162508 | 51 | 34 | 0 | BASE002 | 51470084 | | 986493 | 20160128162512 | 51 | 4097 | 71 | ? | 50866250 | | 986496 | 20160128162513 | 51 | 17 | 62 | BASE001 | 50925666 | | 986499 | 20160128162514 | 51 | 34 | 67 | BASE002 | 51099375 | | 986503 | 20160128162518 | 51 | 17 | 62 | BASE001 | 50558750 | | 986505 | 20160128162519 | 51 | 34 | 66 | BASE002 | 51220667 | | 986509 | 20160128162523 | 51 | 4097 | 69 | ? | 50872542 | | 986511 | 20160128162524 | 51 | 17 | 62 | BASE001 | 56337416 | | 986513 | 20160128162525 | 51 | 34 | 67 | BASE002 | 51593667 | | 986517 | 20160128162529 | 51 | 4097 | 69 | ? | 50970792 | | 986519 | 20160128162530 | 51 | 17 | 62 | BASE001 | 50963292 | | 986522 | 20160128162531 | 51 | 34 | 67 | BASE002 | 50827750 | | 986525 | 20160128162535 | 51 | 4097 | 73 | ? | 50890708 | | 986527 | 20160128162536 | 51 | 17 | 62 | BASE001 | 51484250 | | 986530 | 20160128162538 | 51 | 34 | 67 | BASE002 | 50931292 | | 986534 | 20160128162541 | 51 | 17 | 62 | BASE001 | 50594000 | | 986535 | 20160128162542 | 51 | 34 | 67 | BASE002 | 50980791 | | 986540 | 20160128162546 | 51 | 4097 | 72 | ? | 51010167 | | 986542 | 20160128162548 | 51 | 17 | 63 | BASE001 | 51528708 | | 986544 | 20160128162549 | 51 | 34 | 67 | BASE002 | 50809334 | | 986547 | 20160128162552 | 51 | 4097 | 71 | ? | 50872875 | | 986549 | 20160128162555 | 51 | 34 | 67 | BASE002 | 50916209 | | 986553 | 20160128162558 | 51 | 4097 | 72 | ? | 50984750 | | 986555 | 20160128162559 | 51 | 17 | 62 | BASE001 | 51448333 | | 986556 | 20160128162601 | 51 | 34 | 66 | BASE002 | 51154834 | | 986561 | 20160128162604 | 51 | 17 | 62 | BASE001 | 50753459 | | 986564 | 20160128162605 | 51 | 34 | 66 | BASE002 | 51103166 | | 986567 | 20160128162608 | 51 | 4097 | 72 | ? | 51131958 | | 986570 | 20160128162610 | 51 | 34 | 67 | BASE002 | 50811833 | | 986573 | 20160128162613 | 51 | 4097 | 72 | ? | 50949958 | | 986575 | 20160128162614 | 51 | 17 | 62 | BASE001 | 50951500 | | 986578 | 20160128162616 | 51 | 34 | 67 | BASE002 | 50817292 | | 986581 | 20160128162619 | 51 | 4097 | 71 | ? | 51125791 | | 986584 | 20160128162620 | 51 | 17 | 62 | BASE001 | 51112375 | | 986587 | 20160128162622 | 51 | 34 | 66 | BASE002 | 51024833 | | 986590 | 20160128162625 | 51 | 4097 | 71 | ? | 51030625 | | 986592 | 20160128162627 | 51 | 17 | 62 | BASE001 | 51125542 | | 986594 | 20160128162628 | 51 | 34 | 67 | BASE002 | 51297333 | | 986598 | 20160128162631 | 51 | 4097 | 71 | ? | 51078250 | | 986600 | 20160128162632 | 51 | 17 | 62 | | 51089167 | | 986603 | 20160128162634 | 51 | 34 | 67 | BASE002 | 52381583 | | 986606 | 20160128162637 | 51 | 4097 | 72 | ? | 51176375 | | 986609 | 20160128162638 | 51 | 17 | 62 | BASE001 | 51410792 | | 986612 | 20160128162640 | 51 | 34 | 66 | BASE002 | 51416959 | | 986614 | 20160128162643 | 51 | 4097 | 72 | ? | 50910833 | | 986617 | 20160128162644 | 51 | 17 | 62 | BASE001 | 51024708 | | 986619 | 20160128162646 | 51 | 34 | 67 | BASE002 | 51024458 | +---------+----------------+------+--------+------+---------+-----------+ 51 rows in set (0.03 sec) mysql> Bye ** 27 Jan 2016 Not feeling well, so I didn't go to the Vermont Beef Producers annual meeting. Meanwhile, the base stations pumped out around 50K records (each) to the basedata table between midnight and 8am. Had to change pingtags.c to not bother doing the NI and DB commands if the CH command failed. There was just too much bad data being generated. Also had to put in the correct parameter to sizeof() in xbee_xcv_CH(). Added nanosecond resolution to pingtags.c; the old microsecond resolution can be had with -dUSEMICROSECONDS=1. Now we have to let this run for a while. Having problems getting bbb2 and the direc1 antenna working. direc1 was working for my laptop, wonder what's going on... Meanwhile, pingtags keeps hanging for no obvious reason. I didn't have stdout redirected when it was running in the background, so I restarted it with that set and lets see if it hangs again. I hope it not something obscure like ssh is running out of resources or something. ** 26 Jan 2016 Got bbb2 talking to the nanostation and uploading data to the server (simon.codemeta.com) over the internet! I had to set the nanostation (nano1) to be an AP on the 192.168.1.1/24 network, and the bullet (omni2) to be a bridged station. Discovered that I can ssh from one Ubiquity device to another. But can't get into bbb2 - probably because AirOS doesn't support ECDSA keys. (I get the motd and get kicked off.) Once on the AP's ssh, I have to look at the DHCP leases file to find the IP address of the BBB. Then I 'ssh debian@'. However, since the above doesn't work, I discovered that Port Forwarding works. (However, I should configure the BBB's with static IPs to make this more long-term.) On nano1, I go to Network>Port Forward and add: Private IP: 192.168.1.95 (from the DHCP Leases) Private port: 22 Source IP/Mask: Public IP/Mask: 192.168.56.65/32 (nano1's IP address) Public port: 8095 (the last 3 digits are the 192.168.1.*'s last octet) Note that you'll have to wait a minute or two after you save the changes so that the router/AP have a chance to reset and reload. Then use 'ssh -p 8095 debian@nano1.localnet' to get access to bbb2. Did something similar for the gkrellm monitor on bb2/db... But using port 19150 and its forwarding 9095 port. Wish I could get gkrellmd running on a nanostation! And saved the config file nano1-XM-68725136FD62.cfg. ** 25 Jan 2016 Got the nanostation configured and temporarily mounted on the 3rd floor of the barn. Temporarily because this will be the unit that attaches to the remote field's landowner's network. ** 24 Jan 2016 Well, the "new" pingtags stopped logging data around 3am with: 20160124033603 Sending AT command 'DB', for tag '0x1001' failed: -23/Stale information was used during this function call, retVal: 0/No error. 20160124033607 Sending AT command 'CH' for tag '0x1001' failed: -17/A timeout occured, retVal: 0/No error. 20160124033607 Tag '0x1001' failed. Continuing... 20160124033609 connection to remote radio 0x1001 failed: -20 (An item already exists by that name or identifier) Proceeding because of the EEXISTS error... 20160124033609 Sending AT command 'CH' for tag '0x1001' failed: -22/There is currently no free frame ID that can be used for the request, retVal: 0/No error. 20160124033609 Tag '0x1001' failed. Continuing... I discovered the local connection to the XBee wasn't being xbee_conEnd'ed because there was an error when it was opened. So I'm now freeing it regardless of errors. Although, come to think of it, none of the base stations got anything through to the DB server after the above error, so this seems to be some systemic problem. But I'm not so sure now... Also decided to output a messages if there are any outstanding packets for the connection. Also got a reply from the Digi Community about my congestion issues: http://www.digi.com/support/forum/53109/congestion-with-many-xbee-pros-in-close-proximity-each-other Question: I've got about a 70 XBP24 (XBee Pro 802.15.4) units that I would like to deploy on some moving objects (think ATV's and horses). They will be using the API to communicate to each other (this is many-to-many, but not using broadcasting - just simply 16 and 64 bit addressing in API mode). Do I need to worry about collisions or congestion with this number of XBee's? Especially if they all get talking at once... If so, will packets just get dropped, or are errors returned if there is a collision? Answer: It does not really matter the total number of node so much as it is the total amount of data being sent. As long as you do not exceed the streaming limit of 80kbps, you should be fine. With that said, if a data collision occurs, automatic retries should kick in. Otherwise you should get a response in the API status frame indicating if it was received or not. If it was not received, you may want to re-transmit the packet. But that is dependent on your software protocol if that is necessary or not. So, it looks like I don't have to worry about things. But this is odd - I've got the same pingtags running on bbb1 and bbb2. The one on bbb2 is throwing errors like: 20160124114241 Sending AT command 'CH' for tag '17' failed: -13/An invalid argument was provided, retVal: 0/No error. 20160124114241 Tag '17' failed. Continuing... 20160124114242 connection to remote radio 17 failed: -20 (An item already exists by that name or identifier) Proceeding because of the EEXISTS error... 20160124114242 Sending AT command 'CH' for tag '17' failed: -13/An invalid argument was provided, retVal: 0/No error. 20160124114242 Tag '17' failed. Continuing... But bbb1's errors are like: 20160124114507 Sending AT command 'DB', for tag '0x1001' failed: -23/Stale information was used during this function call, retVal: 0/No error. 20160124114508 Sending AT command 'NI' for tag '0x1001' failed: -25/Transmission failed (check the retVal), retVal is 4/Unknown error.... 20160124114511 Sending AT command 'NI' for tag '0x1001' failed: -25/Transmission failed (check the retVal), retVal is 4/Unknown error.... This is weird. Why is bbb2's pingtags throwing "invalid argument" errors, and bbb1's pingtags not!?! Maybe I should just powercycle both. And I was planning on working on the hardware today! Well, I restructured the TagProcessor routine in pingtags (broke it up into smaller routines), and I'm not seeing the "remaining" errors any more. But I need to run it for a while. Added a command line option too. Made a kit and will propogate it to the base stations. Also made the various wires/cables I need for putting the boxes with antenna on the mounting poles. But I still have to modify the boxes. ** 21 Jan 2016 Pingtags: Found that xbee_conNew can return -20 (An item already exists by that name or identifier), but seems to still have a valid connection data structure, so I'll continue and use it. Also added -w to pingtags so users can specify the "INTERCYCLEWAITTIME" (the time between cycling through the tags file). This will probably help with airwave congestion, but I think we'll eventually need something more sophisticated. ** 20 Jan 2016 Well, installing the RTS/CTS lines on the XBEE doesn't seem to help. I checked 'stty' and it has crtscts set and ixon and ixoff cleared. ** 19 Jan 2016 Dot Perkins came by this morning and I showed her the hardware. Let her know that what I'm still doing is testing. She said "I'm almost testing". At least she knows I'm making progress. I think the problem I'm having with pingtags going catatonic on me is related to the lack of hardware flow control on the ports. I need to set one up (tomorrow?) and see if it helps. I found the pinouts to use today. In the process of diagnosing the catatonic pingtags, I added a lot of debug statements in the code and ran it with gdb and XBEE_LOG_LEVEL=-1 and PINGTAGS_DEBUG=4. Which is how I came to the conclusion that its flow control issues... LibXBee doesn't appear to be getting full messages in API mode. ** 17 Jan 2016 Discovered the last data logged in the basedata database table was for 9 Jan 2016 at 21:48. Investigation indicated there were errors in all the pingtags.err files of all the basestations. The XBee's are returning the error "-22/There is currently no free frame ID that can be used for the request" when it attempts to do a CH command. I changed pingtags.c to output a timestamp with the error message. Since the program has been running for quite a while, I suspect something isn't being freed... Just need to find out what. (I suspect that, since there are errors because sometimes the tags can't be reached, I'm neglecting to free packets when those errors occur.) Also, at some point BASE002's (ID 0x22) name got corrupted. I just fixed it using XCTU. ** 16 Jan 2016 Tested my POE circuit today (for the base stations), and it looks good. So I'll be taking down the base stations one-by-one and installing the circuit... But not until the waterproof RJ45 adapters arrive - some time next week. Sigh. At least I can make some more circuits. And this means the forums I read about the Bullets running on 12 VDC were correct! ** 15 Jan 2016 The big battery on bbb2 is still going strong at 12V. The others are fine, but have A/C power attached. Looked at the database on simon. Found about 5000 (out of around 75,000) GPS records with invalid data, so I deleted them. They were marked with either latitude=0, or active="V". They were spread out between all the bases, but not at the same times, which I find a bit curious, but am not going to look into at this point. Meanwhile, I've spent the last few days getting familar with the Ubiquity radio hardware. So far, the results are really good, and I think they'll make dealing with very large fields a lot better - especially for real-time data collection - as long as they don't drain the batteries. Also did some research on UWB - ultra wide band. Although what I read indicates locating is relatively easy, today's implementations of it only have a reach of about 10 meters! After doing some statistics on the basedata, I've discovered the numbers are all over for both RSSI and deltatime! Looks like statistics are going to be necessary no matter what I do. Also, it appears that obstructions tend to skew the data. From 100,759 records, I'm seeing the following averages - note that none of the basestations or tag moved during this... mysql> select avg(rssi),avg(deltatime) from basedata where remote=4097 and base=17; +-----------+----------------+ | avg(rssi) | avg(deltatime) | +-----------+----------------+ | 57.3055 | 51347.0937 | +-----------+----------------+ 1 row in set (0.04 sec) mysql> select avg(rssi),avg(deltatime) from basedata where remote=4097 and base=34; +-----------+----------------+ | avg(rssi) | avg(deltatime) | +-----------+----------------+ | 55.9663 | 51326.0230 | +-----------+----------------+ 1 row in set (0.04 sec) mysql> select avg(rssi),avg(deltatime) from basedata where remote=4097 and base=51; +-----------+----------------+ | avg(rssi) | avg(deltatime) | +-----------+----------------+ | 71.4455 | 51598.5674 | +-----------+----------------+ 1 row in set (0.04 sec) mysql> select count(*) from basedata; +----------+ | count(*) | +----------+ | 260487 | +----------+ 1 row in set (0.03 sec) mysql> select count(*) from basedata where remote=4097; +----------+ | count(*) | +----------+ | 100759 | +----------+ 1 row in set (0.00 sec) A massive chimney is between bbb3 (base=51) and the tag (remote=4097), plus a floor, but no walls. The other two bases have just walls between the tag and base. bbb2 (base=34) has only 1 wall and about 15 feet. bbb1 has 2 walls, and a floor. I think its time to move some of this outdoors to wide-open-areas. ** 8 Jan 2016 Spent a lot of time reading about GNSS/RTK (and RTKlib) - Real Time Kinematics - which is what "big ag" is using in their precision agriculture. I can't find out much more because a lot of their details are hidden in what I suspect are proprietary protection mechanisms (their own definitions of commonly used words in the industry). I may need to play with this a bit more, but I also might have to buy proprietary hardware - but I don't want to. ** 7 Jan 2016 Spent some time trying a merge in a git. Not sure if I was successful or not. See my DayLog entry for details. Also got some good info from the database; unfortunately, the rssi info is more accurate that the time-of-flight (deltatime). But at least I got some real data. I'm going to let it run for a while (days) so I can get a bulk of data to try some statistical analysis on. ** 5 Jan 2016 Discovered that the GPS needs a real antenna attached if there's an antenna cord on the GPS unit. Sigh. Also, the GPS coordinates are wrong. Discovered that the NMEA coordinates are degrees and decimal minutes times 100. Gotta convert that to something more rational. Sigh. ** 3 Jan 2016 Spent the last couple of days intermittently working on the SARE project... Specifically getting the system to boot and just start working. Finally acheived a kit that would do that last night. I'll spend part of today testing it. Fixed typo in sendgps.sh and made another kit. ** 28 Dec 2015 Well, BBB1 has been running with the new pingtags and updating the database. I'm seeing deviations of over 50,000 nanoseconds in stationary tags (and the other basestations). Still not sure if these deviations are from the OS, or speed-of-light. At some point, I'll need to start doing the real-time calls to eliminate the Linux Operating System latency and interruptions. But Carole is saying I have to write a report soon. And I have to update all the other basestations with the new kit and start collecting data from them. (This database is going to get big.) I also need to modify gpscap to use a different table on the same remote database. Loaded the new kit onto bbb2 and bbb3 and rebooted. Things seem to work now. BBB1 ran out of juice today - probably because of all the compiling, kitting, and debugging I did on it yesterday. I've got it back on the charger - I tested it and it only had about 7 volts before putting it on the Battery Tender. Hope I didn't run it dead. ** 27 Dec 2015 Started adding remote database support... /home/debian/bin/senddata.sh on the base station, and sare-db-update on the server. Note that I had to add the base station's user public keys to the server and severely restrict access to sare-db-update on the server. And sare-db-update should probably institute some additional restrictions in the future for security reasons. I also created a database on the server (saretracker), and a table to receive the -l output from pingtags (basedata). The user is "msf". I also had to change pingtags.c to do logging properly, and without commas. And I had to move the station/tag name to the end of the line because that may not be set in all cases. I need to make a kit with these changes. At the moment I'm reluctant to make a kit for the server because it could reveal too much information about how I operate the server, and I don't want to do that until I'm confident about the security aspects of this project. Hmmm. As soon as I put the pingtags in the background and logged out, I started getting errors from ssh (/usr/bin/ssh-askpass). So looks like I have to create a password-less key and install it in cmsimon. Sigh. Hmmm. Or use keychain. ** 25 Dec 2015 I had two base stations on one battery (the old ATV battery) and when I came home, both base stations appeared active (lights on and blinking normally), but weren't communicating with the network. This morning, both base stations were dead, so I suspect last night's lack of communications was because there wasn't enough power in the battery to power the wifi in both stations. This is an interesting power-loss scenario and one I wasn't anticipating. ** 22 Dec 2015 Well, both the batteries on the base station and ear tag died last night. So that regulator is pretty good at keeping things going for a long time. I also suspect both batteries is now completely drained (which means neither will hold a charge). Meanwhile, found and corrected a bug in scantags (it wouldn't continue looping until it found all tags). Also moved the gettimeofday in pingtags to run earlier so that we can exit "real-time mode". ** 21 Dec 2015 Got all the logic changes to support getting the RSSI into pingtags. Unfortunately, I have to use 'N/A' as the tag ID because there doesn't appear to be a way to get the 'node name' from a node any way other than via the ND command (which is what scantags uses). Well, just discovered that the NI command (when sent to the remote) will probably return the tag's name. Added the NI functionality. Unfortunately, it doesn't always work, so there's a lot more error output. I'm leaving it in the code though. Note that its unreliability has caused me to increase the general robustness of the code. Also, the battery on the eartag seems to last about a day, and then starts to plummet dramatically. I can't ping it from upstairs, but the base station that's 8 feet away still can. And the RSSI (signal strength) tends to drop, but its hard to be sure... It started out at 84 yesterday morning, but I was walking around with it. Right now its at about 49 to 50, and hasn't moved all night or today (and its about 8 feet away from the tag). Oh, and BBB3wl is still running on that battery without any indication of giving up. But its not doing much. ** 20 Dec 2015 Got a new USB Wifi antenna (one that sticks outside the box). Also used 'apt-get update' and 'apt-get upgrade' to get the latest kernels. Now Wifi works! Also got the enclosures for the base stations, but they're going to need some modifications, and I need to wait until the drill press arrives (the gift from Dale) so I can make them. Also need to wait until the power jacks and plugs arrive. However, bbb3[wl] has been running for several days from the (dead) YUASA ATV battery I had laying around! I'm not going to have time to get "disconnected operations" to work before the end of the month, (the logic of sync'ing remote files with a remote database will just take too long to code and debug), so this will be a phase-2 item. Also figured out how to get /sys/devices/bone_capemgr.*/slots updated at boot time - turns out most of the documentation on the 'net are for Angstrom, and BBB rev C has Debian, so I've been mislead by the old documentation. I need to create an /etc/init.d/msf file so /usr/local/bin/msf will be run when the system boots. Also discovered that I don't need wifi-reset any more (because that was required by Angstrom, not Debian). So I'm creating new kits today and testing. Hopefully I'll get around to modifying the boxes. Also got one ear tag built and a station box. Started them running around 14:45, recording to bbb3wl:~/foo.bar. Now I'm running the test to see how long it runs for. (I'm sure the station battery will outlast the eartag battery.) FYI... Hot glue is NOT conductive (whew)! Started some earnest testing. Uggh. And it looks like the XBee's don't support multiple ND queries from more than one host. Sigh. Looks like I have to parcel this out some way, which also means the base stations have to be corrdinated somehow. (This is while running scantags). They symptom is that the xbee's just hang, and never recover. Or go back to using pingtags (which I should probably do since I wasn't getting timing info back with scantags, but I'll have to add RSSI info (ATDB) to pingtags). Made logic changes to pingtags. Using '-d /dev/ttyUSB0 -a -l pingtags.log -v -t cattle.tags' (while cd'ed to MSF/Projects/FarmHacks/Network/MSFCode). ** 16 Dec 2015 Tried moving the Edimax Wifi dongle closer to the Unifi WiFi antenna (6 inches away), and it still didn't work, so I'm now looking for other devices. ** 15 Dec 2015 Tried getting Wifi to work on bbb3 with an EdiMAX EW-7811Un (Realtek RTL8188CUS) dongle (because it doesn't stick out from the board much). Having lots of trouble getting it to work. Tried updating the kernel, but the /opt/scripts/tools/update_kernel.sh got a "Not Found" error. I now have to try getting physically closer to the antenna (but if I do that, then the dongle is useless due to distance). I also need to work on power to the bbb's. The SRM says "The connector used is a 2.1MM center positive x 5.5mm outer barrel. The 5VDC rail is connected to the expansion header. It is possible to power the board via the expansion headers from an add-on card." I'm thinking I can plug the 5V regulator directly into the expansion slots instead of dealing with getting a compatible plug. ** 14 Dec 2015 Spent some time last week at a Maker "conference" sponsored by BovControl. They made a battery case for me, which greatly facilitated getting to the next step (testing the ear tag). Unfortunately, we discovered the batteries lasted on 30 seconds. So I have to look for a new battery set. Meanwhile, I've been spending a lot of time making scantags work in a loop and send its output to a server (via URL). Almost complete. Kathryn Downey was going to Manchester today and got the CR123 batteries today (from Batteries Plus). These are the Duracell DL2/3A equivalents. They work! (I don't know how long they'll last, but they're lasting a lot longer than the coin cells!) I'm thinking that having scancodes update via http/ftp/... may not be a good idea. I'm now thinking I should just dump it in a database - even if its sqlite3. Nope... outputing to a pipe is best - most flexible for all but performance. This means we don't need libcurl anymore! ** 7 Dec 2015 I ran into this problem yesterday, but didn't get it fully diagnosed... It looks like there's a problem with one of the XBee Pro's - specifically the one on bbb3. Every time its connected, the BBB won't power up. Turns out the problem was that I was off-by-one on the pin connections on the XBee. Fortuantely, I didn't blow anything. Discovered that $PGTOP is a proprietary sentence that can be sent out by some of the GPS's. I modified gpscap so that it would be ignored. Also discovered that LibXbee won't handle XBee's unless they are in API mode, so I suspect I have to write a program to set the XBee in API mode when the system is first created. Sigh. Design problems... Its becoming apparent in the testing I'm doing that we can't rely on the radio timing (time of flight). The time of flights are all over the place. I have one module that physically next to another (within 9 inches), and it inconsistently (but more than 50% of the time) has measured time-of-flights that are longer than modules more than 20 feet away! This also tells me that there's quite a bit of latency/nondeterministic-ness to the XBee modules! However, the RSSI (Received Signal Strength Indication) from the scantags command seems to have sane signals. I won't be able to get high resolution distances from these, but it's better than the data I'm seeing from measuring time of flight. Sigh. Also, the battery holders for the ear tags arrived - they weigh more than the radio tag does! However, they didn't include pin connectors. Maybe I should just buy the XBee adapters from AdaFruit, which have the pins and voltage regulators. And, now that use time is nearing, I'm beginning to think it might be better to use a Internet URL as a destination for output than files and/or databases. Of course, this means installing libcurl. ** 6 Dec 2015 When setting up the third Beaglebone Black basestation (bbb3), I discovered that you can't have all the peripherals (network, GPS, and XBee) connected. I suspect this is because the multiplexed I/O hasn't been set up. Also, the time is wrong. So we have to set up the time zone. And the kit is way too big. Looks like I have to drop the kit files (which is OK since the user should have it anyway). And drop the .git directory. ** 11 Nov 2015 When testing the kit, I discovered some problems with settagid. Turns out the MY command takes binary data instead of character data when using the API. I don't remember seeing this in the API documentation, so I'm not sure if this has to be done to the other commands in settagid or not. Only testing will tell for sure. (Not sure how to test the PAN or CHAN as it will disconnect the device from the XBee network.) ** 3 Nov 2015 Got scantags and settagid working. Have discovered that XCTU's values don't match those returned by scantags. Here's the scenario: 0. Start up xctu and get a full network map. 1. 'scantags -d /dev/ttyUSB0 -lva' shows me all the tags and the base station. 2. 'settagid -d /dev/ttyUSB0 -r 0x0013A20040B6A1E9 -n TAG111 -w' changes the tag's NI to TAG111 without error. 3. 'scantags -d /dev/ttyUSB0 -lva' shows the tag with its new TAG111 name. 4. xctu doesn't show the new tag name. 5. Ah Ha! It shows the change if I force a reload of the data by clicking on the "Arrows in a Circle" icon. So, what happens if I power down the tag? Changes are lost unless the -w option is used. ** 12 Sep 2015 I've updated the xbeeat tool to allow for remote AT command execution. This has allowed me to send a command to a remote xbee (an eartag), and for it to send a response. This is now a sufficient full-duplex path for me to get the time-of-flight on. Now to get back to work on pingtags.c... ** 10 Sep 2015 I've been struggling with a design issue in the XBee's. They are designed so that they don't communicate with *any* other XBee unless that XBee's address is specified with a ATDL command. This works for sending data out, but the remote can't send the data back (because it doesn't have a PC attached that can create commands). This creates quite a problem. Will now try to see if I can get the remote to send something (anything) using the API. Turns out the API can get information from the remote xbee (Remote Command Request, and Remote Command Response). Gotta figure out how to do this and update xbeeat to allow it to be done. Then have to figure out how to capture the timestamp on the transmission and reception. ** 10 Aug 2015 Just verified that libxbee is having similar problems on bbb1. But I will have to run some of the included tests just to make sure. debian@beaglebone:~/MSF/Projects/FarmHacks/Network/MSFCode$ ./libxbee-test-0.simple_at tx: -17 txRet: 0 Yup - -17 is timeout - same problem as on db.localnet. So its not a simple 64-bit unclean problem. Time to do some reading on the 'net... NB: Had to change the Makefile to support the "clean" target - which will remove all objects and executables so they can be re-built on a different machine type. Sigh. Looks like there might be some additional communications problems with the libXBee author because Google Code is being shut down. At least he appears to have moved it to gitHub, but all the prior bug reports have not been moved. Just got the test working on bbb1 - turns out I had to have ATAP1 set to turn on API mode. ** 5 Aug 2015 Spent day or two trying to get libXBee to build. Finally got it "working" (i.e. building) on db.localnet (my bedroom workstation). Wrote xbeeat to test it and as a general testing utility. But keep getting errors back when trying to transmit... "Transmission failed", "Undefined error", and a few others that don't make sense. And it looks like Digi (the designer/makers of XBee) finally got their XCTU tool working on Linux (rather than MS Windows), and I was able to test the XBees for correct operation. They are operating correctly according to XCTU, so it looks like there's some problem in libXBee - probably not 64-bit clean. So the next step is to try porting the library to BBB (the Beagle Bone Black), and test there - since I'll eventually have to run on it anyway. Note that XCTU will NOT run on the BBB. ** 27 Jul 2015 Worked on the primary logic for the project - pingtags.c - this sends a message to each "tag" (satellite XBee) and waits for a response. The difference between the send and response should indicate the difference in distance. Basically, I got the rudimentary logic working - getting a list of tags and sending a message. However, I need the libXBee library of routines to address each XBee individually using their API mode. ** 1 May 2015 Evidently, the boot process is not well documented (even @reboot in crontab doesn't work), and the software appears to be somewhere in the middle of the process of moving from SYS V Init to systemd. So I wrote /usr/local/bin/msf to enable the serial ports. It has: #!/bin/bash echo BB-UART1 >/sys/devices/bone_capemgr.*/slots echo BB-UART2 >/sys/devices/bone_capemgr.*/slots This must be run at startup with "sudo /usr/local/bin/msf". I can't seem to get it to work reliably on its own - none of the web pages work: * https://billwaa.wordpress.com/2014/10/13/beaglebone-black-enable-all-uart-ports-at-boot/ * http://tenderlovemaking.com/2014/01/19/enabling-ttyo1-on-beaglebone.html * http://beaglebone.cameon.net/home/serial-ports-uart * ... Otherwise, 'gpscap /dev/ttyO1' works (note that it doesn't print anything out, but updates the gpsinfo.sqlite3 database). ** Feb 2015 Finally made the decision to not work on the device driver until there's better documentation on the multiplexing I/O for the TI Sitara chips - just too much has been changing between versions of the chip. So, I'm going to work on all the support software, starting with the GPS - as that's fairly well understood. Created gpscap.c, which continuously reads from the GPS and writes to a DB file. This DB file can then be independently queried to statistically determine the closest location of the anchor point. Note that to get /dev/ttyO1 to work: sudo bash echo BB-UART1 >/sys/devices/bone_capemgr.9/slots Also, the MicroSD card has to be ejected from the BeagleBoneBlack to get to my development environment. NOTE: "sudo shutdown -h now" seems to take a while (about 3 minutes) to take affect. ** 19 Mar 2014 From my Daylog for today: * (21:04:12) ... Spent some of this evening working on the equipment requirements for the SARE/FarmHack project: $37.95 * 3 Xbee S1 Pro (for base stations) $113.85 $37.95 * 5 Xbee S1 Pro (for ear tags) $189.75 $10.00 * 3 Xbee Adapters (for base stations) $ 30.00 $45.00 * 2 Beaglebone blacks $ 90.00 $7.95 * 2 Proto plates $ 15.90 $5.00 * 2 Half sized breadboards $ 10.00 $39.95 * 2 GPS (for base stations) $ 79.90 $9.95 * 3 UBEC power (buck) converters (for base stations) $ 29.85 Total: $559.25 All of the above is from Adafruit. ** 29 Nov 2012. Just recording some things in case there's any question later... root@beaglebone:/home/jbd# lsb_release -a Distributor ID: Angstrom Description: Angstrom GNU/Linux v2012.05 (Core edition) Release: v2012.05 Codename: Core edition Note that this beaglebone DOES NOT come with expect or TCL/Tk, so interacting with the serial ports will have to be via a library (which it currently looks like I have to write). Files that need modifying from the original distribution: /etc/passwd (add user jbd - via vipw) /etc/group (add user jbd to dialout group - via vigr) /etc/shadow (add user jbd) /etc/gshadow (add user jbd to dialout group) /etc/sudoers (add user jbd) Also discovered that /sbin/init is a link to systemd; as a result the /etc/rc* scripts aren't used. Instead, /etc/systemd/... is. Sigh. Hummm. Just found http://code.google.com/p/etrack/ - which appears to be a similar project - but funded by NSF to the tune of $338K! ** 26 Nov 2012. Finally had some time to work on the project while in the Islands - see entries in the FarmHack Forum. Turns out I had the wrong documentation for the XBee (I was working with the Mesh version documentation when I had the 802.15.4 version of the hardware). Finally found and downloaded the right version of the docs and it looks like I can do what I want a bit easier. Note: for distance measuring, I'll have to turn off a lot of XBee features (like retries, error checking, ...) But that's doable. Note that in the meantime, I've been able to write some software for testing and made a few discoveries that are documented on the FarmHack forum and in some mail messages with GNHLUG. ** 16 Aug 2012. Finally got the hardware. I've researched the Zigbee S2 (didn't get the PROs it turned out), and have a lot of questions. Recently got the GPS, and am attempting to attach to the beaglebone. Also read most of the docs. Looks like I'll need to solder the battery holder on, but I think I can power it from the breadboard for now. I'm a bit concerned that I don't have an antenna for it. Can't much documentation on the antenna. I've ordered an antenna and adapter cable. ** 16 May 2012. Bruce and Louis met at MIT for the "Hacking Embedded Linux: More hardware than you require" (http://www.blu.org/cgi-bin/calendar/2012-may) talk. Talk covered current status, and what hardware Louis needs to produce a prototype. Louis indicated he needed: * 2 Zigbee PRO S2 (http://www.sparkfun.com/products/10421) * 2 Beaglebones Some other things covered in the talk were: * Raspberry Pi ($30 Linux Desktop) * DreamPlug (replacing the Guru Plug) * Cotton Candy from Fxitech.com - a thumbdrive sized Linux system (with video & USB) * Freescale. ** GPS Info The datastream from the GPS is 9600,8N1. Its on /dev/ttyO1 and uses BBB pins as follows: 3.3V - P9/3 - Provides power to the GPS GND - P9/1 - Provides ground (power) to the GPS TX - P9/24 - The GPS's Transmit channel (This is where data is output). RX - P9/26 - The GPS's Receive channel. EN - P9/19 - The GPS's Enable circuit - useful for connecting to CTS on a UART to prevent data overruns. The data coming across the line (from GPS) is one stanza/line, and the first characters of the stanza is the message type: $GPGGA - Time, position, and fix type data $GPGSA - GPS operating mode, active satellites $GPGSV - GPS satellites in view and their ID, elev, azimuth, and SNR $GPRMC - Time, date, position, course, and speed data. $GPVTG - Course and speed info relative to ground The following information was derived from "FGPMMOPA6C - GPS Standalone Module Data Sheet Revision V0A" from GlobalTop Technology Inc. *** $GPGGA - Time, position, and fix type data Example: $GPGGA,064951.000,2307.1256,N,12016.4438,E,1,8,0.95,39.9,M,17.8,M,,*65 Message ID ($GPGGA) GGA protocol header. UTC Time (064951.000) - the current time hhmmss.sss Latitude (2307.1256) - the current latitude ddmm.mmmm N/S Indicator (N) - the hemisphere indicator N=north or S=south Longitude (12016.4438) - the current longitude dddmm.mmmm E/W Indicator (E) - the hemisphere indicator E=east or W=west Position Fix Indicator (1) 0=Fix not available, 1=GPS fix, 2=Differential GPS Fix Satellites Used (8) Between 0 and 14 satellites can be used to obtain a fix HDOP (0.95) - Horizontal Dilution of Precision MSL Altitude (39.9) - Antenna Altitude above/below sea-level Units (M) - MSL Altitude is in Meters (not Feet) Geoidal Separation (17.8) - Geoids separation Units (M) - Geoidal Separation is in Meters (not Feet) Age of Diff. Corr. () - Seconds of differential correction when DGPS is used DGPS Station ID - Not documented in manufacture's spec, but appears to be an integer. Checksum (*65) - Checksum of message - End of line *** $GPGSA - GPS operating mode, active satellites Example: $GPGSA,A,3,29,21,26,15,18,09,06,10,,,,,2.32,0.95,2.11*00 Message ID ($GPGSA) - GGA Protocol header. Mode 1 (A) - Either M (manual - forced to operate in 2D or 3D mode), or A (automatic - allowed to automatically switch between 2D/3D) Mode 2 (3) - Either 1 (Fix not available), 2 (2D... < 4 SVs used), or 3 (3D... >= 4 SVs used) Satellite Used (29) - SV on Channel 1 Satellite Used (21) - SV on Channel 2 ... Satellite Used () - SV on Channel 12 PDOP (2.32) Position dilution of precision HDOP (0.95) Horizontal dilution of precision VDOP (2.11) Vertical dilution of precision Checksum (*00) - Checksum of message - End of line *** $GPGSV - GPS satellites in view and their ID, elev, azimuth, and SNR Example: $GPGSV,3,1,09,29,36,029,42,21,46,314,43,26,44,020,43,15,21,321,39*7D $GPGSV,3,2,09,18,26,314,40,09,57,170,44,06,20,229,37,10,26,084,37*77 $GPGSV,3,3,09,07,,,26*73 Message ID ($GPGSV) - GGA Protocol header. Number of messages (3) - Ranges from 1 to 3 - there may be multiple messages if there are more than 9 satellites used for tracking. Message Number (1, 2, 3) - Ranges from 1 to 3. Satellites in view (09) Satellite ID (29) - Channel 1 satellite - ranges from 1 to 32 Elevation (36) - Channel 1 satellite's elevation in degrees (Max: 90) Azimuth (029) - Channel 1 satellites azimuth in degrees (Range 0 to 359) SNR (C/No) (42) - Signal strength in dBHz (Range 0 to 99). ... Satellite ID (15) - Channel 1 satellite - ranges from 1 to 32 Elevation (21) - Channel 1 satellite's elevation in degrees (Max: 90) Azimuth (321) - Channel 1 satellites azimuth in degrees (Range 0 to 359) SNR (C/No) (39) - Signal strength in dBHz (Range 0 to 99). Checksum (*7D) - Checksum for each message - End of line *** $GPRMC - Time, date, position, course, and speed data (Recommended Minimum Navigation Information) Example: $GPRMC,064951.000,A,2307.1256,N,12016.4438,E,0.03,165.48,260406,3.05,W,A*2C Message ID ($GPRMC) - GPS Protocol header. UTC Time (064951.000) - hhmmss.sss Status (A) A=Data valid, V= Data not valid Latitude (2307.1256) - ddmm.mmmm N/S Indicator (N) - Hemisphere for latitude; N=north, S=south Longitude (12016.4438) - dddmm.mmmm E/W Indicator (E) - Hemisphere for longitude; E=east, W=west Speed over ground (0.03) - speed in knots Course over ground (165.48) - course in degrees (0=north?) Date (260406) - ddmmyy Magnetic Variation (3.05, W) - NB: Not available without custom chip. degrees, E=east, W=west Mode (A) - A=autonomous mode, D=differential mode, E=estimated mode Checksum (*2C) - Checksum for each message - End of line *** $GPVTG - Course and speed info relative to ground Example: $GPVTG,165.48,T,,M,0.03,N,0.06,K,A*37 Message ID ($GPVTG) - GGA Protocol header. Course (165.48) - Measured heading in degrees. Reference (T) - True Course () - Measured heading in degrees. Reference (M) - NB: Not available without custom chip. (M=magnetic) Speed (0.03) - Measured horizontal speed. Units (N) - Measure of speed (N=knots) Speed (0.06) - Measured horizontal speed. Units (K) - Measure of speed (K=kilometers/hour) Mode (A) - A=autonomous mode, D=differential mode, E=estimated mode Checksum (*2C) - Checksum for each message - End of line ** XBee Info Useful web sites: * http://cpham.perso.univ-pau.fr/WSN/XBee.html - general info about connecting to an XBee for the first time. * https://learn.adafruit.com/xbee-radios/overview - excellent overview and useful info on the XBee. When an XBee is connected to an Ubuntu 14.03 LTS system on a USB port, the following will appear in /var/log/syslog: Jul 19 10:46:17 db kernel: [153266.958408] usb 5-2: new full-speed USB device number 2 using uhci_hcd Jul 19 10:46:17 db kernel: [153267.156331] usb 5-2: New USB device found, idVendor=0403, idProduct=6001 Jul 19 10:46:17 db kernel: [153267.156337] usb 5-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3 Jul 19 10:46:17 db kernel: [153267.156341] usb 5-2: Product: TTL232R-3V3 Jul 19 10:46:17 db kernel: [153267.156343] usb 5-2: Manufacturer: FTDI Jul 19 10:46:17 db kernel: [153267.156346] usb 5-2: SerialNumber: FTG90KYF Jul 19 10:46:17 db mtp-probe: checking bus 5, device 2: "/sys/devices/pci0000:00/0000:00:1a.2/usb5/5-2" Jul 19 10:46:17 db mtp-probe: bus: 5, device: 2 was not an MTP device Jul 19 10:46:18 db kernel: [153267.956794] usbcore: registered new interface driver usbserial Jul 19 10:46:18 db kernel: [153267.956809] usbcore: registered new interface driver usbserial_generic Jul 19 10:46:18 db kernel: [153267.956818] usbserial: USB Serial support registered for generic Jul 19 10:46:18 db kernel: [153267.968543] usbcore: registered new interface driver ftdi_sio Jul 19 10:46:18 db kernel: [153267.968557] usbserial: USB Serial support registered for FTDI USB Serial Device Jul 19 10:46:18 db kernel: [153267.968659] ftdi_sio 5-2:1.0: FTDI USB Serial Device converter detected Jul 19 10:46:18 db kernel: [153267.968687] usb 5-2: Detected FT232RL Jul 19 10:46:18 db kernel: [153267.968688] usb 5-2: Number of endpoints 2 Jul 19 10:46:18 db kernel: [153267.968690] usb 5-2: Endpoint 1 MaxPacketSize 64 Jul 19 10:46:18 db kernel: [153267.968692] usb 5-2: Endpoint 2 MaxPacketSize 64 Jul 19 10:46:18 db kernel: [153267.968693] usb 5-2: Setting MaxPacketSize 64 Jul 19 10:46:18 db kernel: [153267.971436] usb 5-2: FTDI USB Serial Device converter now attached to ttyUSB0 Easy ways to communicate directly with the XBee: * cu -l /dev/ttyO2 -s 9600 dir * minicom ** libxbee Info This is information about the library used to interface with the Xbee's. Basically, you start by calling xbee_setup (or xbee_vsetup) to initialize internal data structures. You need to pass it a pointer to an address to store the internal data structures. This address is used in subsequent calls to the library's routings to identify the context of the library. Resources created by this routine are freed by xbee_shutdown. Then you can call xbee_conNew to create a connection to an XBee hardware module. This requires the pointer set up by xbee_setup, and it returns (in the 2nd parameter's address) a pointer to a data structure used to maintain the state of the connection. The 3rd parameter (char * type) can be one of: * "Local AT" - To change and get the status from the local hardware * "Modem Status" (Read only) for status updates from the hardware. * "Transmit Status" (Read only ) if you want status responses. * "Remote AT" - To change and get the status from a remote Xbee. * "Data" - 2-way data exchange with a remote module. * There are others, refer to the xbee_conNew man page. Then call xbee_conTx to send an AT command to the hardware. If this returns a non-zero value, then an error occurred. Otherwise, the command succeeded. The parameters are: 1: address of the xbee_conNew data structure. 2: Address of a character buffer to receive the AT command response. 3: The AT command to send. For example: "BD", "NI", ... Then call xbee_conRX to receive the response from the AT command that you sent. The parameters are: 1: address of xbee_conNew data structure. 2: Address of the packet structure (struct xbee_pkt *). This will be allocated and filled with information by the xbee_conRX routine. 3: If not null, then an int pointer to a count of the remaining packets. The data read is in the packet at the 2nd parameter. When all communication is complete, then call xbee_conEnd to close the connection, and xbee_shutdown to release all resources for the Pconnection. Instead of calling xbee_conRx, one can use xbee_conCallbackSet to retrieve data; it appears to create the packet and populate it with data when it becomes available. ** Non-standard software on the BeagleBone Black The following sofware was installed on the BB Black after it was purchased. This software is needed for developing the FarmHack "Item Locator" software: * lrzsz, minicom - Needed to communicate across the serial lines. * samba-common, winbind, cifs-utils, keyutils, smbclient - Needed to access MSF's development environment * sqlite3 libsqlite3-dev - Provides the database software. * libxbee - GitHub: (HTTPS: https://github.com/attie/libxbee3.git/, SSH: git@github.com:attie/libxbee3.git) *** /home/debian/MSF/Projects/FarmHack/Network/LibXBee All this files in this directory were provided by Mr. Attie and are publicly available. See https://github.com/attie/libxbee3. This was previously sponsored by the Google Code project. ** MSF created files Files that were created by Miles Smith Farm for this project are described in the subsections below. *** /usr/local/bin/msf - enables the serial ports for this BBB. Contains: #!/bin/bash echo BB-UART1 >/sys/devices/bone_capemgr.*/slots echo BB-UART2 >/sys/devices/bone_capemgr.*/slots Note that this file must have execute permission, and must be sudo'ed. It should be run only once at boot time. *** /home/debian/MSF/Projects/FarmHack/Network/MSFCode All the files in this directory were created by Bruce Dawson at Miles Smith Farm exclusively for this project (and possibly its successors). ** Testing Hints Use minicom for testing. You will have to set it to 9600N1. Use the following minicom commands to see what you're doing: ^A-O-Screen and keyboard-P ... Adds a linefeed ^A-O-Screen and Keyboard-Q ... Turns on local echo A good alternative is to use the command 'cu -l /dev/tty... -s 9600 dir' Use '~.' to get out of it. Note: You may have to download the cu package to use this command. ** Future Features Some features that future versions of this project can work on are categorized in the subsections below: *** Hardware Features that require changes in hardware are described in the subsections below. **** 3 GHz Oscillator RTC A oscillator and counter that operates at 3 GHz should, theoretically give us sub-inch resolution. This will probably change some software requirement too. **** Battery Monitor The base stations should probably have a power cape that would monitor their power source. **** Long Distance WiFi Ubiquity makes some Wifi broadband devices that can reach up to 15 Km (15 Km bi-directional!). These would be a good way of getting information to the server quicker for a more real-time experience of watching cattle move on the web site. **** A base-station aggregator I believe some people will NOT want these on the Internet... They'll want to access the data without use of the internet. This would best be accomplished by (another) BBB that doesn't have an XBee or GPS on it, but would have a hard drive and some access method (probably private network). It would do everything that simon does now (run a MySQL server and a web site). The owner would access it from their local network. *** Software Features that require changes in software are described in the subsections below **** Put --version in all software Make sure the user can query the version of all software utilities we ship. **** Base Stations could upload to each other If only one base station has network access, it is (theorically) possible for the base stations to use their XBee's to upload their databases to adjacent basestations until all the data reaches a basestation with WiFi/Network access, which can then upload the data to the server. The XBee Pro's seem to have a greater range than WiFi. **** Use PPS from GPS To upgrade the base stations to Tier-1 NTP servers. This would give us far more precision in their location. **** Need some "auto-configuration" software Its kinda hard to configure everything... Between building the basestations, eartags, and possibly network, you then have to configure all the above with XBee ID's and names. It would be nice if a lot of this could be auto-configured and sorta made idiot-proof with nice error messages and diagnostics. **** Use logrotate Use logrotate to "rotate" the pingtags.err and gpscap.err files, along with sendgps.log. **** Base station holds data and release when an event occurs In rural areas, signals are not reliable. If the base station can't reach the remote server, then it should store the data until it can reach the server. This is non-trivial, because records have to be deleted when they are transmitted - which usually means a database operation with transactions and multiple writers. SQLite3 can't do that. Have to come up with my own way or find something like a "cache" that can do it. *** System Changes in the general architecture of the system are described in the subsections below **** Use syslog for logging errors Using Linux's syslog facility for logging errors (like communication interruptions), instead of the current "write to a redirected file", would allow people to send the errors to a central server and decrease the per-base maintenance requirements (especially to control disk overflows). * Emacs variables These must be kept near the end of the file: Local Variables: mode:outline mode:auto-fill End: