Raspberry Pi online logging thermometer

RPi_DHT11
Source: http://www.uugear.com/wordpress/wp-content/uploads/2014/01/RPi_DHT11.jpg

My office never has a steady temperature; sometimes it is too hot, too cold, or just flip-flops all day long. To prove that I wasn’t crazy, I decided to try a DIY thermometer project with a Raspberry Pi that currently was not doing anything else. The best part for me is that this was a project that nicely combines hardware with software in a very practical manner; from wiring together circuits, collecting data in a database, to the end result of a fairly nice looking, online interactive chart.

I have a Model B, but for the most part, these instructions should be generic enough for any of the raspberry pi’s. I think at some point the GPIO board was expanded, but that only added higher-order pins, not affecting the lower numbers. Also, I did not want to use a breadboard, nor any soldering – in the end, I twisted the wires together and taped them down for a layman’s compact design.

Goal

Make a device that records temperature every 15 minutes with as few wires as possible, able to save data on a database and display a public, online interactive chart to summarize data

My supplies:

Setting up the Pi

Sometimes the Raspberry Pi SD cards come pre-loaded with NOOBS. I had an older project still on my card, and I knew I wanted a headless version (no GUI, no pretty windows, pure command-line). I found a few online articles that helped guide me with this (like https://www.raspberrypi.org/forums/viewtopic.php?f=91&t=74176), and I also removed a number of things like Wolfram Mathematica, CUPS, and anything else I did not want. I then set up the wireless adapter so I could login with no wires (since that was part of my project goal), and then hardened the OS (again, google was my friend). I could still do more to harden it, but for now I am satisfied.

Wiring the temperature sensor

temp_sensor_GPIO_layout
Source: http://i.stack.imgur.com/5EKzW.png

I found and used a few different online articles to help with this part. I cannot find the one that I used for the DS18b20 Temperature probe, but the basics are very similar to this one: http://www.circuitbasics.com/raspberry-pi-ds18b20-temperature-sensor-tutorial/

Since I did not want to use a breadboard, nor did I want to solder, I used the female-to-female jumper wires. In retrospect, I could have used something like a female-to-bare-wire jumper. I took one end of the jumper I had, stripped off the female jumper, and twisted that to each end of the DS18b20 probe bare wires. I also twisted in the 4.7k resistor per the instructions (between yellow and red). Some kind of in-line wire clamps would have been nice, but I did not have any at the moment (and did not plan ahead for them), so I twisted the wires and used electrical tape to secure and wrap them up nicely in a low-profile bundle.

Next I plugged the female jumpers into the GPIO board as instructed; red (power) to pin 1, black (ground) to pin 6, and yellow (data) to pin 7.

The first couple times I tried to use modprobe and see the raw data from the Pi, I was foiled because of bad connections (yeah, twisting wires together is not a great way to make sure they stay together). But after redoing my wiring, I finally got some good data from w1_slave. Note also that each 18b20 will (should) have its own id, so if using more than one probe you will have multiple device directories, each with a w1_slave file.

Raspberry-Pi-DS18B20-Temperature-Sensor-Tutorial-DS18B20-Raw-Output
Source:
http://www.circuitbasics.com/raspberry-pi-ds18b20-temperature-sensor-tutorial/

Data Acquisition

At first, I thought about running a sqlite database and webserver locally on the pi itself. But then I got to thinking, if the IP address changes (I don’t have a reserved name), then trying to publish the new URL would be tricky. Since I have access to public-facing servers, for me it was easier to simply collect the raw data on the RPi and send it to another server to publish web pages.

After a few trials and errors, I finally came up with a script that does the following:

  1. scrapes data from every w1_slave file (I only have one right now)
  2. parse out the sensor id and the raw temperature reading (will be Celsius * 1000)
  3. ssh to webserver
  4. insert data directly into sqlite database on webserver
$ cat /home/pi/getTemp.sh
#!/bin/sh

# set -x

host="some_host"
login="some_login"
creds="$login@$host"
out="/home/pi/ssh.out"

if [ ! -e /sys/bus/w1/devices/*/w1_slave ]
then
 exit 1
fi

set `ls -1 /sys/bus/w1/devices/*/w1_slave`
num=$#
while [ $# -gt 0 ]
do
 thisDate=`date +%s`
 id=`echo $1|cut -f6 -d"/"`
 temp=`tail -1 $1|sed 's/.*t=//'`
 echo "thisDate:$thisDate id:$id temp:$temp"

 if [ $temp -gt 0 ]
 then
  query="insert into temps values ($thisDate, '$id', $temp*1.0/1000);"
  ssh $creds "echo \"$query\"|sqlite3 piTemps.db" > $out 2>&1
 fi

 shift
done

And then schedule it in cron:

00,15,30,45 * * * * /home/pi/getTemp.sh > /home/pi/getTemp.out 2>&1

The database

I build a query that inserts the data into a sqlite database called piTemps.db. To create the database and table:

sqlite3 piTemps.db 'CREATE TABLE temps(unix_time bigint primary key, celsius real);'

The unix_time is stored as a large integer (the number of seconds since January 1, 1970, aka epoch time). My insert statement multiples the temperature reading by 1.0 to convert it to decimal, then divide by 1000 to get the order of magnitude correct.

Graphing the data

This turned out to be the most time-consuming part. I had a pretty clear idea of what I wanted my end product to look like, but getting there was not easy (at least, for me). I have prior experience with a js graphing library called Highcharts and I wanted to use that for my presentation. (I have also used d3js, which is awesome, but a steeper learning curve.)

I started with the PiThermServer as my prototype (shout out to Tom Holderness for putting this on github). I like the idea of a simple NodeJS webserver that reads data from a sqlite database and displays data via Highcarts. Making small changes to his base code, I was able to get a simple line chart. I don’t know if you have ever seen a line chart of inside temperature data, especially one that is updated every 15 minutes – it is pretty boring. 🙂 I knew I wanted a radial infographic that resembles those big outdoor thermometers; I finally landed on a dual-guage speedometer example from Highcharts, which I adapted for temperature (fahrenheit and celsius).

To get the most recent temperature for my thermometer, I use this query:

SELECT unix_time*1000 unix_time, round(celsius*9/5 + 32,1) fahrenheit
FROM temps WHERE unix_time =(select max(unix_time) from temps) limit 1;

The subquery gets the maximum unit_time from the database. The outer query then gets the temperature that is associated with that maximum time. When I was testing, I was doing a number of other things as well. Now that the query is pretty much set, a faster way to get the same data might be:

SELECT unix_time*1000 unix_time, round(celsius*9/5 + 32,1) fahrenheit
FROM temps ORDER BY unix_time desc limit 1;

Note the classic conversion of Celsius to Fahrenheit. Since I have a dual-guage chart, I don’t need to convert it here anymore, but I left in for posterity as well.

I wrap that in a js call from my nodeJS server.js:

   var current_temp = db.all("SELECT unix_time*1000 unix_time, 
           round(celsius*9/5 + 32,1) fahrenheit 
           FROM temps WHERE unix_time =(select max(unix_time) from temps) 
           limit 1;",
      function(err, rows){
         if (err){blah blah blah}
         data = {temps:[rows]}
         callback(data);
    });

temperature_guage.png

The hardest part was tweaking the JSON configurations for Highcharts just the way I wanted. You can see my final result here (as opposed to copying it all into this blog post):
http://onyx.csit.parkland.edu/~cschultz/PiThermServer/temp_angular.txt

Next, I really like the Highstock viewer of showing a day, week, month or year of data, along with the overall summary of everything. So I spent a few hours adapting and perfecting that as well:
http://onyx.csit.parkland.edu/~cschultz/PiThermServer/temp_all.txt

The call to the database was slightly different, since I wanted all the data:

SELECT id, (unix_time-21600)*1000 unix_time, round(celsius*9/5 + 32,1) fahrenheit
FROM temps ORDER BY id, unix_time ASC;

And finally, here is my end-product:
http://onyx.csit.parkland.edu:35008/

temperature_end_product.PNG

Overclocking

I spent a good 6 hours today learning about and testing (mostly the latter, reboot after reboot) overclocking. Experience has always been my best teacher – although I would not say it is always the most fun of instructions.

For starters:

After reading a slew of google hits on overclocking and sometimes using the terms above, I learned a few things.

  1. Heat is bad.
  2. Good to have the RAM-to-FSB ratio by 1:1
  3. Need more Core Voltage for the CPU to reach the really high speeds
  4. Sometimes as the FSB increases, one has to make the CAS timings “looser”
  5. Using tools like TAT, CoreTemp, CPU-Z and Orthos seem to be common to the overclocker crowd

So first I had to think about heat and airflow. I did a bit of research in the days prior and found a nice cooler that I like that will eventually cost me $35 ($10 discount + $15 mail-in-rebate). I am a bit cynical on how much heat it will actually displace. I got to thinking about convection and heat transfer and air flow. My case currently has two fans that force air out of the box on two non-opposing sides, a large vent right over the CPU that allows air to come in, and a stock fan that whirls that incoming air onto the heat sink and aluminum fins. If the outgoing vents get clogged up with dust (which happens to a greater degree than I imagined, than you have practically zilch new cold air coming in. Which is bad. Not only does my CPU generate heat, but I have four hungry harddrives (two of which are quite warm), and gas guzzling video card and memory sticks. These all contribute to a general state of “not cold”. All that heat building up inside the case makes the heat fins and CPU stock fan rather useless. So I took off a side panel and suddenly the average CPU temps dropped 6 degrees Celsius. How is that for a heat sink! 🙂 I also reduced the case fan speeds to their lowest settings and am thinking about just removing them completely (reduction of noise).

Most of my time was spent tweaking one of the many parameters in my BIOS and rebooting to see what kind of effect it had. Sometimes my computer would not boot, sometimes it crashed while loading Windows, sometimes it crashed shortly after loading Windows, and sometimes crashed during a test of Orthos. Very few times everything worked smoothly. Specifically at the very beginning before tweaking, once in the middle during an “AHA!” moment, and then at the end when I finally gave up and took some screenshots of where I am stopping today.

It doesn’t help that it was incredibly hard to match up my BIOS with things I was hearing online and in the forums. For instance, there is no setting for the RAM:FSB ratio (and to be even more accurate, I believe it is the FSB:DRAM ratio). On top of that, my BIOS has a setting for the DRAM clock speed, but the absolute lowest value is twice that of the FSB. So I was extremely confused. During the first half of my testing, I was noticing that my FSB:DRAM ratio was consistently 3:5 if I left the BIOS setting as AUTO. And then I finally stumbled upon an article that explained how the Ratio halves the value that is specified in the BIOS. Well, if only the BIOS writers could make that a bit more clear!! So I finally found that if I set my DRAM clock to FSB x 2, the sun shone upon me.

That was my half way point; I had the FSB up to 320 MHz, 60% above the base clock speed of 200 MHz, taking the CPU clock from 1.8 GHz to about 2.88 GHz. Not too shabby at all. But I wanted to push 3 GHz. Especially if I have a nice fan coming in, I should be able to push the envelope a little and find out what happens.

I quickly hit a wall, though. I could do a FSB of 333 MHz (which comes in at just shy of 3 GHz), but a FSB of 343 MHz just gave me all kinds of problems. It was frustrating. I could do the FSB at 333 and alter the DRAM to 666 and leave everything else at auto and have it working. But 343…. no way, man. I tried to change the CAS latencies, I tried to add more power to the CPU, I tried to play with things that I had no idea what they were (FSB Termination Point Voltage, Row Refresh Cycle, PCIE speed). I rebooted, tested, rebooted, crashed, rebooted, didn’t POST, rebooted…. again and again. After a while I was cursing Orthos for not telling me why it was crashing (or windows for that matter). The online documents told me that I should add more power to the memory, but my BIOS has locked it at Auto, and no amount of Googling or phone calling could land me a practical way to unlock it (tried, Ctl, Alt and Shift F1 in many different combinations).

 

So I called it quits with 333 MHz. I am not displeased with that. I just want more. MOAR!!! 🙂

 

Orthos had me up to 62 Celsius after a few minutes and I don’t want that to happen often. If my fan does not keep it down closer to 55 under load, I am going to return. Or throw it out the window.

I am rather convinced CPU-Z has errors; Allendale? 1.288V??
I am glad I was able to keep the timings "tight" at 333 MHz
62 degrees! Not awful, but not something I want long term

I shrugged

Just watched “Atlas Shrugged” and I still don’t get the title. Huh? While I won’t go as far as Roger Ebert’s official review, I also found this piece a bit of a big question mark – and not one that really encourages me to find out more. I see that Ayn Rand has another film getting ready to hit the silver screens and now I am much more disinclined to see it.

My initial curiosity about the idea of Government becoming too big, too oppressive and too ridiculous is what drove me to even consider this title. But even if I were to attempt to superimpose a dose of reality unto this concept, I don’t see how “going on strike” and removing the innovators helps anyone. Where is the incentive for those going on strike? If the pursuit of happiness is really the rock-bottom goal of life (which I do not agree with. At all), then how are these people fulfilling their purpose while sucking air?

There are some loose associations with the Christian Rapture, in which God pulls all his people out of the world (ala “Skyline”). If all the “good” people are uploaded (talk about offshore!), then what’s left? This film attempts to ask the same question, and kinda thumbing of the nose, a sticking out of the tongue. The main character, a sister co-CEO of the railway giant, gets it right when she says John Galt just gives up – everyone who joins that group has taken the easy way out. It sounds like a lose-lose situation, and I firmly believe that Ayn Rand’s Objectivism ultimately leads to this conclusion.

Other than the problems with the underlying philosophy, I found that my belief was not suspended at all. James and Dagny not only don’t see eye to eye, but they are somehow able to take thier monolithic company in almost two different directions?!? How do Dagny and Hank Rearden have time to cavort all over America, and how do they travel? They “tested” a new metal on a new track crossing a new bridge breaking a new US train speed record with…. wait for it, a passenger train pulling over 25 cars?!? I’m sorry, but there is just too much that removes itself from me to make this film either entertaining or educational.

I have decided that I do not like this theme

The current theme is “Coraline”. I chose it because I wanted something that had a nice Gallery option. And something simple. I must have messed up somewhere. I do not like the font (which I can probably change if I dig deep enough), and by default there is no easy way to “press” a page (again, probably a way to change that). There is something that just seems intrisically wrong about this theme.

A post on Heather’s facebook

if only I could cross-post to both places at once…. oh wait, there is an app for that.

Anyway.

“I just finished the book; my not-yet-fully-formed thought is that the underlying issue is not so much “how much pink is too much?” but rather how we are being programmed.

I was the parent who thought it would be a fascinating to train a child that what we commonly know as “red” to be “green”. Just as an experiment. See what the doctors and teachers would make of that. For some odd reason, I never gained full support for that idea (can’t imagine why, think of all the research possibilities, all the fun, all the… what, not fun you say?).

However the programming that Peggy Orenstein attempts to elucidate is much more sinister in my opinion. No matter what your personal thoughts are on sex and religion (oh my, social faux pas right there, using those two words in the same sentence!), Big Business is attempting to brainwash all of us simply to generate more revenue. Money is the bottom-line driver for so much in this country, and people have figured out what sells. Even more insidious, those same people have figured out how to create a cycle in which they define what sells. It is our job to fight that onslaught, that injustice, day in and day out. Am I being too melodramatic?

The trick, and the focus of writing such interesting books and articles as Mrs. Orenstein has, I believe, is to figure out how to do that. Hence Heather Grush Schultz’s desire for a book club and http://free2bgirl.prophpbb.com/. There are other aspects of the book which intrigue me as well. For instance, Mrs. Orenstein’s handling of sexuality and identity is something that has practically become taboo, but we all seek freedom to express our thoughts and learn who we are. And determine, for ourselves, who we are.”

MLK Celebration “Striving for Genuine Brotherhood”

Recently heard about this event, going to go check it out. Also, Nathaniel Banks has been talking about Mo Betta, and I want to see what all the hype is about. 🙂

http://ci.champaign.il.us/2012/01/03/2012-rev-dr-martin-luther-king-jr-countywide-celebration/

 

Update 14-Jan: NG put up a review: http://www.news-gazette.com/news/people/2012-01-14/king-speaker-civil-rights-leader-pursued-genuine-brotherhood.html

Not liking the textbook

http://www.amazon.com/Launching-Imagination-2D-Mary-Stewart/dp/0077379802/ref=sr_1_4?ie=UTF8&qid=1326311924&sr=8-4

 

What good is a blog if I cannot gripe, eh? =)

Aside from the typos (one right at the top of the 2nd page!), after reading the first chapter I felt like my imagination fell off the launch pad with a loud crashing noise. Ironically, I feel more creative now as I trash the book then when reading it. *grin*

 

As I was reading the preface, something was nagging me at the back of my head. While I appreciated what was being said, something was rubbing my fur the wrong way. And then I realized, I don’t like the style, the layout! Hah! For a Graphic Design textbook, the book seems garish to me. The Table of Contents is horrible! No spacing for page numbers or anything. And this is my first impression. The Introduction launches us into a barrage of images; images are visually stimulating, much more so than words, so I appreciate that. But as soon as she said “Contemporary Art” another red flag went up. I have zero appreciation for Contemporary Art. I don’t care about painters, sculpters or metalsmiths.

Which made me realize, wow, I have a closed mind. The Introduction tries to tell us the unique value of new beginners because they are, like children, ready to explore the world and let curiosity lead them. I am starting to feel old now.

And while the images certainly lend a sense of excitement, I found them quite distracting from the text, making it hard for me to follow what the author was trying to say. Worse, as I read the captions, I started to disagree; “Surrounded by explosive energy?!? Ha, that is just a sloppy painting”. The last sentence of the Introduction says that if I use my time well, I can really get my rocket off the launch pad. We will see….

And with that I jumped into Chapter 1, “Basic Elements”. I guess we have to start somewhere. First element is Line. Not “the line”, or “a line”, but ….. line. I rolled my eyes when I read “the inherent dynamism of line is embodied in the first definition [a point in motion]”. Whatever dude. But then I thought about it. A point, in and of itself, does not really define a dimension. In a single dimension (which would be described by a line), I would contend that the point needs yet another dimension (namely, time) in order to “travel”. It may exist at any point in time on that line in that single dimension. But to travel…. So I disagree with two of the four definitions used by the author. But I like the idea of a point in motion. I’ll have to hold on to that.

We then learn about other qualities (or attributes, if you will) like Orientation, Weight, Direction and Continuity. It is good to understand what the author thinks of these things in order to lay a framework for the rest of the book. Yet again, I was distracted by a picture and its caption; “… Newman used just two lines to express both spiritual strength and human fragility”. Where do they teach this mumbo jumbo? I see something totally different. Like, one dark heavy line and one implied line with lots of fuzzy hazy things around it.

Which led to the next section, different kinds of lines (actual, contour, calligraphic, organizational, implied, and linear networks). I like the concept of closure, found under implied lines – I have used that myself (as you can see from my trial doodle). I really disliked the example of Caravaggio’s The Deposition to portray closure in contours – are you kidding me, he uses color to emphasize contour all over the place.

 

I didn’t really get the section on “Using Line”. Maybe because it is subconscious to me? If I want a line, I use a line. Using the previous pages, everything is a line, which obfuscates (or rather, abstracts) this discussion even further.

Key Questions (these are from the book – I am merely suppling my answers)

  1. I do not feel that I have a dominant orientation to my lines. I use them all. I might have to answer with the default “horizontal”, since most of my world is with computers and text.
  2. When lines are repeated or intersect, they can create emphasis or contention. I mean, where is this question going?
  3. How would the composition be changed if one or more line were removed? What composition? Any composition in general? It would depend on the composition – Newman might be moved to tears, and for that alone I would consider it. 🙂
  4. The last “question” asks me to consider using line to direct attention to compositional importance. But everything is a line. What would “compositional importance” look like with no line?

 

Next section is Shape. I make no promises to leave my cynic’s hat at home. 🙂