tony_osp: (Default)
Saturday, December 31st, 2016 12:00 am
Some time ago I started experimenting with IoT (Internet of things), using primarily Arduino. This blog is a simple diary of my experiments.

As the base project for my experiments I'm working on the SmartGarden - an intelligent multi-station irrigation and environment monitoring
system.

Initially I started from just irrigation part, using OpenSprinkler design, later on moving to the more advanced Sprinklers_pi (running on Arduino Mega) firmware, adding to it more and more features and capabilities. As I was adding more and more new things - multi-station support using RF network of nodes, sensors data collection and logging etc, I've rewritten large parts of the code and changed hardware design.

Currently the project reached v1.0 release (running my home irrigation system since May 2015). The system is stable and reliable.
It is still work in progress, with some fit-and-finish required (WEB UI improvements etc), but it is good enough to use.

Also I'm working on the next generation of hardware and firmware (v1.5). As of mid-Oct 2015 v1.5 is nearly-ready, I will be upgrading my setup to it before next watering season starts.

Source code for my project (both hardware schematics and firmware source) is available on GitHub: https://github.com/tony-osp/SmartGarden.

tony_osp: (Default)
Sunday, May 22nd, 2016 09:08 pm
Last month I replaced my SmartGarden controllers with the new v1.6 version, this includes both new hardware and new software.

SmartGarden v1.6 is still based on Arduino, but on Moteino Mega version - which has twice the amount of RAM, as well as RFM69 RF module. Controller hardware was redesigned around Moteino Mega MCU, also I added few sensor probes ports to the Master controller. Ethernet controller is now based on W5500 WIZNet chip, which is a lot faster than the standard Arduino Ethernet shield based on W5100.

Controller is still mounted in a plastic box, with DHT21 sensor (top right) and RF antenna visible outside.

SmartGarden Master controller - outside view

I just replaced the main logic board (on the right side), leaving relay boards (left top) and voltage converter module (left bottom) intact.

SmartGarden Master controller - open

Local LCD display is the same 16x2 large-font module, with 4 control buttons. But local LCD and buttons are useful mostly for setup and local channels testing, all standard operations are done using WEB UI.

Firmware was also upgraded to the new version, with vastly improved UI and new features.



Home page shows an overview dashboard, including current weather (obtained from Yahoo Weather API), next scheduled watering event, water usage for the last 7 days, and current sensors readings. Menu provides access to schedules, sensors dashboard, historical sensors data, watering logs etc.
Sensors and hardware config (number of stations etc) are defined in a "device.ini" file, and are imported on "factory reset". Customization (e.g. Zone Names) is done via UI.

Outdoor temperature reported on this display is the reading from a temp probe (thermistor placed outside), remote temperature 1 and 2 are readings reported by two remote (connected via RF) stations, as well as humidity readings. Soil moisture is obtained from a special hardware soil moisture sensor connected to one of the remote stations, and Garage humidity/temperature are read from DHT21 which is part of the Master controller.

Right now I have Master station (with 16 channels) + two remote stations in use, connected via RF link. I'm planning to add one more remote station later this summer, at the far end of my property.

So far the system works pretty well. I did watering zones water consumption calibration (using flow meter), and now the controller accurately calculates water consumption of each zone and can graph it by dates, by zone etc.

The source code is available on my GitHub: https://github.com/tony-osp/SmartGarden

Future Plans

I started working on v2 version of the system, moving Master controller to Raspberry Pi MCU. Arduino is cool, and works great for remote nodes, but it is really hitting the limits of RAM and flash, and RPi is a lot better.

Actually I already have the same code ported and running on RPi 3, but it is not a complete work yet. Also I would like to add several more features to the system before putting v2 into real use. Probably will do the switch this fall or winter, after the end of this watering season.



tony_osp: (Default)
Monday, February 15th, 2016 05:14 pm
For the last few months I was not working much on the SmartGarden project. It is winter now, and I have some time before next watering season to finish up all updates.

However last few weekends I had some time and used this opportunity to work on v1.6. SG v1.6 adds the ability to use RFM69 RF module instead of XBee for communication.

The advantage of RFM69 is two-fold - first of all it is way cheaper than XBee 900 modules, but at the same time RFM69 (especially its high-power version) are promising much better range, comparable with latest XBee HP modules at a small fraction of the cost. Also RFM69 is available as a native option on Moteino Mega, which reduces complexity and device size - it becomes possible to fit whole controller (MCU unit as well as RF module) into single small module.

All this is goodness, but the downside is that RFM69 is a simple, low-level transmitter. It does not have all the smarts and processing power of XBee, since unlike XBee it does not have onboard processor. RFM69 is a plain basic transmitter/receiver module, and it must rely on the host MCU for smarts.

Another complication for me was the need to restructure SmartGarden Remote Protocol stack to allow multiple types of RF transport. In the initial version I tried to separate out actual RProtocol implementation from underlying transport, but few unintended dependencies creep in, and I need to clean it up.

I did most of that work over the last few weekends. Now RProtocol is reasonably cleanly separated from the underlying transport, this allows using either XBee or RFM69 (and in theory even both simultaneously), and I also wrote suitable transport driver for RFM69. The driver relies on RFM69 library from LowPowerLab, but I had to add few workarounds to get it all going.

The main thing was to add support for polling mode. Normally RFM69 library relies on interrupts to handle incoming packets, but I found that in the SG Master controller it conflicts with SD card and Ethernet - all of them are sharing SPI bus, and RF module can break in a middle of SPI transfer of Ethernet or SD card. I could've disable interrupts to block RF while Ethernet or SD card are active, but it would defeat the purpose of the interrupt - I might as well pick up RF data in polling loop.

Also despite of the interrupt-driven incoming packets handling in RFM69, it relies on polling to send ACK and handle other duties, and it cannot really do much without polling. In the view of that I just added polling-only mode to RFM69 library and it seems to work fine. The only negative of this approach is reduced bufferisation of the receive path - you have to pick up incoming data sufficiently quickly before next packet arrives (with interrupts support you have one packet buffer). But since senders are using retries and generally expected activity rate on the channel is low, this is probably OK.

As usual, the code is available in the SmartGarden GitHub repo.

I tried few simple range tests with SG controller on RFM69 915MHz link, and it seems to work pretty well. I'm getting about 80db signal at few hundred yards distance, on a small 1/4 wave antennae, and in wet weather. Will try actual max distance I need to achieve in a few weeks.
One remaining bit - I need to add protection from packets duplication.LowPortLab RFM69 library is good and simple, but it does not have any protection from packets duplication on retries. For some of the requests it is kind of OK to receive two of these, but in some cases it is undesirable. I can add smarts to handle it in the SG driver for RFM69. XBee handles it in the XBee itself, with RFM69 you need to do it in the main MCU.

tony_osp: (Default)
Saturday, October 24th, 2015 11:23 am
SmartGarden v1.5 WEB UI is designed first of all for Desktop or Tablet browsing experience. However sometimes it is necessary to open SG UI from a mobile device - such as Phone, for example to start/stop zones when testing sprinklers.

Technically SG v1.5 UI works on mobile, small-screen devices, and most pages resize correctly. Home page is the only one that looks sub-optimal on Mobile screen. I fixed this problem by creating new Mobile Home page optimized for small-screen devices, and added small switcher that automatically selects right type of Home page depending on the browser window size.

The new Mobile Home page looks like this:



It is essentially the same home page, just optimized for small-screen devices. It uses "flow" design, allowing automatic layout re-flow depending on the available screen width. Menu is also removed from the screen to free up space, converted into panel showed on "menu" button click:



On the technical side - supporting mobile devices is actually pretty easy for SmartGarden v1.5 UI, because the UI is written using jQuery Mobile. Just the UI is optimized for Desktop-size use, but it works OK on the mobile devices as well - except Home page, and that part is fixed now.

tony_osp: (Default)
Sunday, October 18th, 2015 06:30 pm
As planned, I just finished converting logging display to the new format, and changed graphing to provide more high-level information.

New graphing tab in "Logs" page shows three basic graphs:

1. Historical watering times for selected time period
2. Historical overall water usage for selected time period
3. Water usage by Zone (again for selected time period)

Water Usage by Zone



I find these graphs more interesting and useful rather than original sprinklers_pi graphs of watering by time of day, or day of week etc, because usually you would want to either see actual log records (and new schedules-based watering log provides it), or you want to see aggregated metrics data, or you want to get insights into which zones are the most "costly" in terms of water consumption. New logs views provide exactly that.

Also I've re-written the visualization UI. Original sprinklers_pi UI was relying on the back-end to filter and package the data for the right view, which makes sense if you have reasonably powerful MCU (and MySql handy), but this approach became royal pain in the neck if you have to rely on low-power Arduino MCU with files-based storage. Good thing is that modern browsers allow offloading data manipulations to the client JavaScript.

New visualization code retrieves basic, plain logs data (the same format as what is used to power other log views), and then produces right information for visualization using JavaScript. Also I changed the graphing library to Highcharts instead of Flot - I like Highcharts powerful and sleek UIs, also Highcharts are used in other parts of the SmartGarden UI and use of the same library speeds up things quite a bit (typically library will be already loaded by the time you hit Logs page).

Overall I'm quite happy with the result.

...
At this stage SmartGarden v1.5 is nearly-ready. The only remaining planned software work is to add support for Thermistor-based temp sensor, but other than that it is done (barring possible bugs etc). Adding thermistor support is not hard, it is essentially a variation of the Analog temp sensor SmartGarden already supports, main extra piece is to add translation table - typical thermistor requires rather complex non-linear conversion to produce temperature reading, and you have to use either expensive floating point expression or translation lookup table. But conceptually it is fairly easy to do.



tony_osp: (Default)
Sunday, October 11th, 2015 06:57 pm
As the part of the code rework in v1.5 I changed the way how watering logs are stored and visualized.

Initially in v1.0 I was using essentially the Sprinklers_pi visualization model, just changing back-end to be file-based instead of MySql, however I found two problems with the v1.0 implementation. First of all I found the UI to be inconvenient, because it shows logs by-zone. But for all practical purposes irrigation is scheduled based on "Schedules", and then the schedule includes one or more zones. Most of the time you don't have to track individual zones - usually I would like to know if particular schedules ran and when, and then I might go into details to see which specific zones were running and for how long. However this is not now Sprinklers_pi logs UI was organized.

Second problem was logs rendering performance on systems with large number of zones configured. My v1.0 backend implementation was using one file per zone, and to render complete view the MCU had to read and process dozens of files. On a low-power MCU like Arduino it takes few seconds.

For v1.5 firmware I changed the model. First of all Schedules became the primary top-level logging construct. Logs UI now shows the list of schedules that ran, and details (when, duration etc). This view is extremely fast (backend specifically optimizes for it). Also I added "Details" logs view, that shows the list of schedules as collapsed items, and when you click on any logged schedule it will open up to show details (which zones were running etc).

Backend implementation also changed, by optimizing storage structure. Now watering logs are stored in two types of files:

- Schedule logs. This file has one record per schedule run, as well as top-level information (date/time, duration etc). One file per year.
- Detailed watering information (down to zones detailization), one file per month.

When rendering default Schedules log view, backend needs to read the first file. This is pretty fast process, because this file has one record per schedule run. Rendering will become a bit slower when reaching the end of the year, but based on simple calculations it should be still pretty fast (and I can move it to per-month files if necessary).

When rendering "Details" log view, backend reads appropriate per-month file, depending on the dates range it may be one or few files. Because default dates range is up to one month, typically backend will need to read one or two files - which is also reasonably fast.

Default "Schedules" logs view:

"Schedules" log view


"Details" log view:

"Details" log view

And of course it is still possible to view and download actual watering log files, they are in CSV and can be easily opened using Excel or other programs (for data analysis etc).

I still need to rework graphics log visualization. I'm not happy with the current visualization, it does not really give much meaningful details of the watering history (e.g. it shows watering by time of the day or day of the week, but it is not very useful). I'm thinking to change it to show historical water usage (by dates), and to show water usage detailization by zone - to easily see which zones are consuming more water than others.



tony_osp: (Default)
Sunday, September 27th, 2015 08:14 pm
My v1.0 SmartGarden system is up and running, it was keeping my lawn and shrubs green this summer, also collecting environmental data. Overall pretty smooth ride so far (fingers crossed :-)).

In the mean time, I was working on v1.5, and it is nearing completion. v1.5 hardware is using more powerful Moteino Mega (1284p) controller, and the firmware changed considerably. New v1.5 firmware has new WEB UI, specifically designed for multi-station irrigation + monitoring system, it has a lot more features and different look&feel. Backend is also significantly changed, to facilitate new features, also I've rewritten core scheduling system. Most of the code now is vastly different from the Sprinklers_pi code, except nntp, tftp and few other small modules.

I'm planning to upgrade my home system to v1.5 later this fall.

SmartGarden v1.5 Home screen

Next Step

I'm planning to switch to the built-in Moteino Mega RF module instead of XBee Pro 900 that is used today. XBee Pro works, however I would like to install additional station at the far end of my property, and at that distance the signal is rather weak. Built-in RF module is supposed to be much more powerful and sensitive. Also it reduces hardware footprint, and new stations based on this design will be cheaper (no need to buy expensive XBee Pro).

I have not decided yet whether to install v1.5 first and then work on switching RF module, or maybe switch RF module before installing new hardware. But either way I'm planning to do it this winter.

Next big step is to port Master controller code to run on SmartPhone/Tablet. This will add a lot more power to the Master controller hardware, and it will give me color, high-quality screen for the device instead of the basic 16x2 LCD. Most of the interaction is happening using WEB UI, however the device itself will look a lot better with the color screen.

This will also enable totally new mode of installation - when the Master controller does not have any zones or sensors connected to it directly (and Master can be installed indoor), but it remotely (via RF) controls actual control and monitoring stations across the property. This is what I consider v2.0 hardware. In theory it is possible and not that hard.



tony_osp: (Default)
Saturday, August 22nd, 2015 08:51 pm
I'm working on the next gen WEB UI for the SmartGarden system, and while debugging javascripts, I noticed some WEB requests perf numbers for the SmartGarden system. WEB Server perf is quite impressive - simple requests like json/state execute in less than 10ms, and 18KB file download (which involves reading the file from MicroSD card and emitting it) is completed in under 120ms.

Taking into account that this is just a tiny Arduino MCU this is actually pretty good. Arduino MCU is doing its part, but it is also clear that W5500 Ethernet controller has pretty low overhead, super-fast TCP handshake, and very fast MCU interface. W5500 was expected to be faster than the old W5100, but actual results exceed my expectations.
Of course, Raspberry Pi or cheap x86 or ARM Tablet will do even better than that, but there is a huge perf difference between 1+ GHz 32bit ARM processor and 16 MHz Arduino CPU.

This super-fast WEB requests handling makes SmartGarden WEB UI fast and fluid, in spite of considerable amount of information on the page. Async requests model also helps a lot, but high perf WEB server is the key.
tony_osp: (Default)
Sunday, August 16th, 2015 08:02 pm
One of the new capabilities I wanted to add to v1.5 system is the ability to monitor water usage. I started by adding support for hardware water meters (of a typical pulse type), and even bought nice water meter device like this one:



It works, but while working on this feature I realized that supporting water meter is not enough. Ideally I would like to know how much water is used by every zone, so I can optimize watering for maximum results while conserving water.

But it is impractical to equip every zone with a separate water meter. It is also hard to use common water meter to measure individual zones - in some cases it may require multiple water meters (e.g. my setup uses three separate water feed connections), also sometimes water is dispensed manually, for hand-watering from the same water line (going through the same water meter).

However there is a way to do it, and actually it does not even require connected water meter :-)
The water flow for each zone is relatively constant when it is running, and if you calibrate each watering zone you can reasonably reliably calculate the amount of water used simply based on the run time of the zone.

To calibrate each zone you can either use common connected water meter (if there is one installed in the system), or you can use simple cheap stand-alone water meter like this one:


It costs only 10-15 dollars, and it allows measuring water consumption. You can just run each zone for 5-10 minutes, measure the amount of water used, and calculate calibration value. Or as a poor-man option you can use your main house water meter - it has dials you can read, just need to make sure nobody in the house is using water while you are measuring it.
Once it is done, the system can be running with no water meter at all, while producing reasonably accurate water usage reports.

Of course, ideally would be to have real connected water meter as well - it can be used to cross-check calculated and actual consumption, also it could be really helpful as a mechanism for detecting broken valves and water leaks. But minimal setup can run without it.

This mechanism for water metering (both computed water usage and actual water meter support) are part of the v1.5 firmware. I'm planning to have it ready for v1.5 system installation this fall - probably end of September or so.


tony_osp: (Default)
Sunday, August 16th, 2015 07:49 pm
Since I have not posted anything for some time, it makes sense to give a quick update.

The SmartGarden v1.0 irrigation controller I installed earlier this year works well so far, pretty reliable and efficient. Keeps my lawn green:-)

I mostly finished new v1.5 hardware, based on Moteino Mega (1284p). This hardware is a bit faster due to W5500 Ethernet controller, and has more RAM (1284p has 16KB), and most importantly - Moteino Mega has built-in high-power RF module. v1.5 hardware still has XBee socket, and I'm using XBee 900 Pro right now (for development and for compat with the v1.0 system), but I'm planning to switch over to the new RF module over time.

I'm planning to swap the Master station controller to v1.5 hardware this fall, after the watering season. In the mean time I'm working on v1.5 SmartGarden firmware.

The v1.5 firmware adds new features and capabilities, including Waterflow counters, analog sensors (intended first of all for soil moisture sensors, but can be any type of sensor), and it also has new WEB UI.

V1.0 WEB UI was still pretty much the original Sprinklers_pi WEB UI, just with few new pages added (sensors dashboard etc), while v1.5 WEB UI is essentially complete rewrite. It is a lot more feature rich, and while using the same core framework (jQuery mobile) it is optimized for bigger screens - a Tablet/iPad/Desktop.

The WEB UI work just started, and may take few more weeks - I can spend only few hours on weekends on the project. But it is progressing well, I will post some screenshots when ready.
tony_osp: (Default)
Sunday, June 28th, 2015 06:34 pm
I installed SmartGarden system at my property few month ago, and was using it daily since then. The system is reliable and efficient, works quite well, keeping my grass and shrubs green. Right now I have 20 zones connected to it, used daily. Also SmartGarden keeps collecting sensors data (temp, humidity, air pressure).

I have additional Remote station completed (8 more channels), but decided to postpone installing it. Instead I'm using it to finish work on the next iteration of the firmware. Also I'm working on the next version (v1.5) of hardware for the Master controller, based on Moteino Mega and W5500 Ethernet chip, the plan is to install new Master controller hardware this fall after the watering season finishes.
tony_osp: (Default)
Sunday, May 10th, 2015 07:55 pm
Last weekend I finished installing and turned on my new SmartGarden system. Actually the Master controller was installed few weeks ago, and I was running few channels on it for testing, but now I have complete system in place up and running. The system consists of:

1. Master controller. It is network-connected, it handles scheduling, logging, sensors logging, WEB interface etc.
2. Remote voltage converter to allow connecting Orbit DC valves to 24VAC irrigation system
3. Remote RF station, driving additional 4 channel of Orbit DC valves, and also handling additional Temp/Humidity sensors

Master controller has 16 channels, out of these 12 are 24VAC and 4 are DC valves-compatible. Out of these 12 24VAC channels 4 are further converted to Orbit DC compatible using remote voltage converter. Remote RF station adds 4 more channels - 20 channels all-up right now. Remote RF station uses 900MHz link and can reach quite far.
Across the system I have 6 sensors - 3 temperature sensors, two humidity sensors, and air pressure sensor. I'm planning to add water flow sensor later on.

I will post detailed report next weekend with some photos.

The software seems to handle all standard duties OK, I noticed few minor bugs (UI-related), but all functionality works. By common standards probably it can be considered "Public Preview" level of quality;-)

Also I have one more 8-channel Remote RF station in works, to add 8 more irrigation channels + additional sensors at the far end of my property, increasing total number of irrigation channels to 28. But probably it will take few more weeks before I can install it.
tony_osp: (Default)
Saturday, February 14th, 2015 12:11 pm
After finishing up few things I published the Master station code on the GitHub. The overall system is at Alpha stage, but is usable - all key functionality works - you can schedule and manually control watering zones across multiple stations, the system will show Sensors status across multiple stations both in Dashboard and in historical/log view.

Sensors data viewing is really more of a prototype rather than finished work, but it works. It was mostly a test UI for me to see actual sensor readings, and I added jQuery Thermometer and HighCharts Gauge for the fun of it :-)




As the next step I'm planning to build few more of Remote station boxes using Moteino MCU, but in totally sealed enclosure, suitable for installation outdoors - to get ready for system installation across my property this spring.

After that I'm planning to start porting the Master station code to a small-screen Tablet. Tablet has a lot better screen than just basic 1602 LCD, and it has WiFi, RTC and other goodies - and with the current super-cheap prices for small tablets (you can find 7" tablet for under $80!) it is hard to find a better value.

tony_osp: (Default)
Sunday, February 1st, 2015 11:48 am
Recently I came across an interesting Arduino-compatible MCU module - Moteino. It is specifically designed for building low-power, RF-connected nodes. Moteino includes AVR chip, necessary power/reset/clk circuits, as well as RF module and even 4Mbit Flash chip. The whole thing is built as a small (1"x2") micro-module. The circuitry is designed for low-power operation.

The most interesting (for me) is the Moteino Mega version - it uses 1280p chip, which provides best hardware specs in Arduino world - 128KB program Flash, 2 serial ports, decent number of GPIO and 16KB RAM - which is actually double of the standard Arduino Mega, and 8x times of Arduino Uno.

Moteino price is also pretty good - base Mega module is under $20, and with powerful (long-range) RF module and 4Mbits Flash chip - about $30. This is actually cheaper than long-range XBee module alone, and I'm considering moving SmartGarden system to Moteino base. Moteino can also work with XBee RF, but if the onboard high-power RF module delivers on the promised specs - I just don't need XBee at all.

I ordered one of the Moteino modules and will be experimenting with it, if it does work as promised - it will be great technology base for SmartGarden.


Moteino Mega
tony_osp: (Default)
Sunday, February 1st, 2015 11:05 am
The multi-station system is mostly working - all core functionality works. I can start/stop/schedule Remote irrigation zones from the Master, can read Sensors connected to Remote from the Master, the system keeps time in sync and updates both Master and Remote UIs when operating Remote zones.

Main missing part is some WEB UI work to create something like Dashboard to show current sensor readings across the whole system, and WEB UI rework to support full set of 64 zones. Currently the back-end supports up to 64 zones, but the WEB UI (which is based on the original Sprinklers_pi UI) can handle only 15 zones, unfortunately simply increasing the number of zones in UI is not possible (due to the way how the UI communicates with backend for certain operations), and partial rewrite is required.

I posted the code for the Remote station on Github, the code is generally working (Alpha-quality).

I have not posted the Master code yet - waiting for resolution for the licensing issue for components created by Richard Zimmerman. Some of the components in the Master (WEB server, Scheduler, NNTP, TFTP) came from the original Sprinklers_pi code, and they are (c) by Richard. In e-mail exchange Richard stated that he intends this code to be freely used for non-commercial purposes, but there is no explicit license attached.
We discussed it with Richard in e-mail, and he said that he intends to attach suitable CC license, once it happens I will post complete Master code.

Next steps: I'm planning to focus on the hardware, building a couple of Remote stations in weatherproof boxes suitable for installation outside, in wet and humid conditions. In parallel I will continue to work on the software, to fix WEB UI and get it to v1.0.

Eventually I would like to move the Master controller code to run on a small-size tablet - it will be a lot more powerful and efficient hardware base. Remote stations should stay on AVR (Arduino).

tony_osp: (Default)
Thursday, January 15th, 2015 06:56 am
I did not post any updates for some time, but I made good progress on the project.

First of all - the controller code now supports multiple stations! Primary use of this capability is to support remote stations (connected via Xbee RF links), but it also supports multiple types channels connected directly to the controller - Parallel, Serial (OpenSprinkler-style etc). All types of interfaces (parallel, serial, RF-connected) can be used simultaneously, in the controller UI they show up as multiple stations.

To support this facility I had to add the notion of Station, list of stations, mapping of Zones to specific stations/channels etc. Also this required lifting restrictions on the number of zones (original sprinklers_pi code supports only 15 zones), as one of the goals of the multi-station support was to handle larger installations.

Remote stations also allow connecting sensors (e.g. temperature, humidity etc), just like the Master controller.

System topology and configuration is stored in EEPROM, but is bootstrapped from an INI file. I.e. now there is an INI file (device.ini) that defines the configuration - types and number of channels connected, pin assignments, types and number of remote stations etc, and on EEPROM reset this configuration is imported into the system. This is a lot better than hardcoded pin assignments of the original sprinklers_pi.

The master controller is still running on Arduino Mega, but I'm planning to move it to HP Stream 7 as the controller. Arduino system resources are really too limited, and this is becoming a problem. (I'm planning to use HP Stream 7 instead of Raspberry Pi because Stream 7 includes 7" touch screen and built-in WiFi, cost-wise it is a lot better, and much better hardware specs than RPi).

Currently on Arduino Mega the Master controller is limited to 64 Zones, and 16 Stations. Number of sensors is not limited, but practically I'm assuming it is within a dozen or so.

This was quite a lot of work, and required re-writing core parts of the sprinklers_pi code. Remote station actually does not use any sprinklers_pi code at all, the Master controller still uses WEB server and few other parts from sprinklers_pi. But when moving to HP Stream 7 I might re-write it as well, because I can take advantage of the core WEB server provided by the platform. But there is also some benefit in having stand-alone, portable code. I have not decided on it yet.
tony_osp: (Default)
Sunday, November 30th, 2014 06:08 pm
One of the problems I had to solve when building Remote station was finding suitable LCD for it. For various reasons I wanted to use RBoard - integrated MCU module with build-in 4 relays and RF sockets, but RBoard creates its own problems.

First of all RBoard has fairly limited number of GPIO pins - it is essentially Arduino Uno-level hardware, with only few pins available. And out of these available pins 4 are already used for relays, I need at least 4 for input buttons, plus I need pins for RF module and for connecting sensors. RBoard just does not have enough of free pins to run standard parallel 1602 LCD.
Second problem is the voltage. RBoard is running at 3.3V, which is OK for broadly available sensors and RF modules, but it puts constrains on LCD module as well.

Each of these two problems by itself is not a big deal, but together they are creating certain challenge. Typical 1602 LCDs are available in 3.3V, but I don't have free pins for it. LCDs with I2C interface are also available, but usually they are running on 5V.

After searching for some time I came across this LCD device.

It is available from both Jameco and Amazon (sold by Jameco), it does what I need and is surprisingly inexpensive for its capabilities.

After trying out the device I'm quite impressed - this LCD is pretty clear and bright, characters are about the same size as a typical 1602 LCD, it runs on 3.3V, and it works via I2C interface. And it also has RGB back light, allowing you to select any desired color of display.
This LCD also has 20 instead of 16 characters in line - which is nice, allowing me to show additional information (e.g. temperature and humidity) in addition to the existing sprinklers control UI.

The module itself is unusual - unlike typical 1602 LCD this one is a single monolithic module. There is no PCB there, all electronics is inside common slim sealed module. I think it is using "Chip on Glass" technology.



The LCD just needs two external capacitors to run built-in voltage regulator, and one resistor for I2C bus - that's all. It is also quite advanced in its capabilities - you can programmatically control contrast, the device allows loading in 16 custom characters (double of a typical 1602), plus it has 20x2 screen. And the price is pretty good - $12.95 at Jameco, with usual volume discounts.

Connection to RBoard is easy - just two wires (plus ground and power). And it is also possible to put other devices (e.g. sensors) on the same I2C bus, so these two wires are shared.

One (minor) thing is that this module has thin pins more like an IC - it is intended for soldering on a PCB rather than directly connecting wires. I ended up using a piece of a prototype board to make the base for LCD, the same board is also acting as a wiring point for input buttons.
For the actual release (not prototype) version of the Remote station I'm thinking about adding I2C GPIO extender chip (can be mounted on the same LCD board), this would allow connecting LCD and input buttons to MCU with only few wires, plus it will allow programmatically controlling back light color (and will free up another 4 pins on MCU).

Another catch in all this - control library. This LCD does not work with a standard LiquidCrystal Arduino library (the controller is different, not just the interface), and it took me half a day to find a library that works. But it was worth the hassle - overall I'm quite happy with this LCD.

tony_osp: (Default)
Sunday, November 30th, 2014 03:40 pm
As I mentioned earlier, I'm working on expanding the Sprinklers smart controller to make it support a network of wireless remote stations. Each station can control a set of watering zones, and it can also collect data from environmental sensors - temperature, humidity, air pressure, soil moisture. Remote station can also collect water flow data.

Over the last month I finished design of the remote station, built a prototype device, and now I'm finishing up the firmware for it. This post is an overview of the Remote station.

Remote station is built on Arduino - powerful yet inexpensive platform for DIY projects. Bit attraction point of Arduino in this context is the availability of all sorts of sensors, RF modules and other add-on hardware, and associated libraries. Also Arduino has low power consumption, and is suitable for even battery-powered devices (although right now this part is not important for my project).

My remote station consists of an Arduino MCU, local LCD and buttons (to allow limited local control), relays for controlling irrigation zones, few sensors, and RF module for communication with the Master station. I'm using excellent RBoard MCU from iteadstudio.com, it includes AtMega328P (Arduino Uno-level hardware), 4 electromechanical relays, SD card slot, and Xbee and nrf24l01 modules interfaces. RBoard is just more convenient (and more compact) for my purposes.

LCD is C0220BIZ-FS(RGB)-FBW-3VM display from Jameco, it provides 20x2 character mode with RGB back light, it connects via I2C interface and runs on 3.3V - perfect fit for RBoard.

As the RF interface I'm using Xbee Pro 900MHz modules - I need sufficient range and power to be able to reach far end of my property in adverse link conditions - foliage, rain (you still need to get sensor readings!), devices location in a wet box near the ground etc. Unfortunately regular inexpensive modules like nrf24l01 cannot do it, but Xbee Pro 900 can. Xbee modules are more expensive, but for a few devices in a home DIY project it does not really matter that much. Good thing is that it is possible to use nrf24l01 modules instead of Xbee, I might experiment with it later.

Remote station supports both regular 24VAC valves, and Orbit DC valves. 24VAC valves are connected directly to control relays, while Orbit DC valves are connected via my voltage converter module (simple circuit that produces DC pulses on On and Off).

Remote station is powered from a standard 24VAC line. It is actually possible to use any suitable power sources (e.g. 12VAC or 24V DC), main constrain is the need to have sufficient voltage to drive irrigation valves.

Remote station is powered by a small rectifier/buck converter module. I could've build one, but there was a suitable module available on Amazon, and I went an easy route:-)


RBoard




The station is assembled in a clear plastic 6"x4" box. The box is reasonably weather-sealed, although I would not risk putting it under the rain - I had to drill few holes for buttons and LCD mounting screws, and it is hard to make it truly watertight.

All together it looks like this:



The empty space at the bottom of the box is the space for the Orbit DC voltage converter - it is not required for 24VAC valves but is necessary for driving Orbit DC valves.

I have baseline software up and running, except Xbee communication part. That code needs more work, and it requires also updates to the Master station (to add Remote stations support).

Next steps - hopefully I will have more time available before the end of this year to finish up the code and to get RF hookup going. Once it is done - it will be possible to start making 3-4 stations to get ready for the next irrigation season.

Another thing I'm considering to try - moving Master controller to a small tablet like HP Stream 7. It has very nice 7" screen, and pretty powerful hardware - all for $99 (!). Initially I was thinking about moving it to RPi, but Stream 7 is a way better machine - it has more resources, nice 7" screen, WiFi, decent storage, even backup battery. And it is running full Windows, which makes development a lot easier.
tony_osp: (Default)
Sunday, November 2nd, 2014 06:52 pm
As I'm working on my Sprinklers/Sensors controller, it is becoming more and more of a SmartHome controller. I just need to add X10 interface to it, to enable interaction with light controls and switches I have around the house to get the controller to that new role.

Unfortunately Arduino hardware is becoming more and more of a problem due to limited resources. RAM is ridiculously low (8K on Mega), CPU is pretty slow (so even parsing few hundreds kilobytes file becomes a challenge), and even simple tasks like driving decent 5"-7" display becomes pretty hard due to overhead.

I was thinking for some time about ways to overcome this problem. On one hand I like (and need) Arduino extensibility, but system resources are just too low. Arduino Due is an option, it is faster and has more RAM - but difference is not really that big. Rasberry Pi or Beagle Bone is another good option - they have way more power, although there are some problems with using sensors and devices that require precise timing - code running under high-level OS like Linux just cannot control IO with good precision.

I also looked at an interesting project called X86Duino, which is an Arduino-compatible board that uses X86 CPU running at 300 MHz. It does not use Linux or another high-level OS, and can provide precise timing, also system resources are way higher than what you could dream of in Arduino land - >20x CPU perf of Arduino, and many orders of magnitude more RAM. Sounds tempting.

But thinking more about it I came across another possible option - I can use a small, inexpensive Tablet as the controller, with Arduino board as the IO extender. There are readily available Android and even Windows mini-tablets with 7" screen for under $100, and this buys you complete system - powerful CPU (at least as good as RPi), 7" touch screen, WiFi, Bluetooth, Audio, camera/sensors and even built-in backup battery. If you simply try to assemble comparable set (e.g. RPi + 7" touch screen + WiFi/Bluetooth) - it will already cost you more than $100.
And to cover IO needs I can use a standard, inexpensive Arduino board - it can handle sensors and devices communication (essentially as an intelligent IO hub), and it can connect to the main device using readily available USB.

Both Android and Windows are good options, Android tablets are slightly cheaper (due to lower specs) but Windows devices have more power. I'm considering both options, but it appears to be a good way to move to a more powerful controller.
tony_osp: (Default)
Sunday, November 2nd, 2014 04:38 pm
Over the last two weeks I was able to spend few more hours on my project, and put together sensors data visualizer using Highcharts and Knockout. It produces pretty graphs like this:
:
Sensors data visualizer


The UI is pretty simple, and is really more of a concept rather than finished product. It allows selecting dates range, sensor, and chart type (Daily, Hourly etc). Graphing is done using Highcharts, input fields and interaction is handled using Knockout.

In the process of putting it together I had to re-write sensors handling backend. Original code worked, but unfortunately performance was unacceptable - it could take up to 16 seconds to fetch the data and redraw the graph. This was caused by having sensor data in annual logs - each sensor was getting its own annual log, and by the year end it could accumulate quite a lot of data - about 17K records (at 30min interval). Records are pretty small, and overall file size was only 260K or so, but unfortunately for a tiny controller like Arduino this was already way too much.

I investigated where time was spent, and found that out of 16 seconds the lion's share (12 seconds) was spent in reading the file (using fgets), and another 4 seconds in parsing records using sscanf. I could've probably bring it down to 4-5 seconds by reading data manually in bigger chunks (e.g. 512 bytes) instead of fgets, and by writing custom parser code instead of sscanf. But 4-5 seconds is still too much for an interactive UI.

To solve this problem I changed the backend code to use monthly files instead of annual files. I.e. each sensor's data is stored in a separate file, one file per month. Because in most cases user wants to visualize last week or last month, it dramatically reduces the amount of data that needs to be processed. Typically the chart will redraw in just over a second, and on the month boundary it may take up to two seconds - not ideal but usable. The time to redraw the graph is proportional to the dates interval, and for a whole year worth of graph you still need to wait for 16seconds, but it expected to be an edge case.

Another possible enhancement is to change the visualizer to download and consume CSV files directly, parsing it in the javascript in the browser rather than Arduino backend. This would allow further improving redraw time by 2x/3x, because direct file download of the original CSV file works a lot faster - no need to parse it on the weak Arduino CPU, also file download uses 512 bytes buffer which helps. But this is a project for another weekend.

Overall I'm satisfied with the Sensors logging and visualizer mechanism, it appears to be fairly reliable and efficient. I'm using it already for over a month, and it was extremely stable.

Next steps

As the next steps I'm planning to add second sensor (DHT21) to the controller, and most importantly - add remote controllers connected via Xbee 900 RF link. This would allow me to have distributed sensors and irrigation controllers network on my property.