How to keep up with new versions of programs

Keeping up with versions on your computer can be quite tedious, especially if those programs do not provide an auto-update feature. You could visit the vendor homepage for each software product you intend to keep up to date and check manually or use a program and hope it knows how to check for updates for your software and last but not least visit a website regularly showing newly released/updated software.

Or you can build something yourself.

The example used here is awful and I am sure there a a million better ways to scrape websites (and as you will see later, can be easily replaced by something else) but should suffice to demonstrate on how to keep track of versions of programs you deem important.

Shown below is a method on how to scrape version information from vendor websites using curl and the usage of some unix commands to modify output from curl. If you want to know what each part in the pipe does just start with the initial command, look at the output and add a step at a time.

Example:

#!/bin/bash
echo -n "Metapad: "
curl -s -m 3 http://liquidninja.com/metapad/download.html | grep -A 5 "Latest Release"  | tail -1 | sed "s/Version //g"
echo -n "Pidgin: "
curl -s -m 3 http://developer.pidgin.im/wiki/ChangeLog | grep version | head -2 | tail -1 | perl -p -i -e "s/<.*?>//g,"  | awk '{ print $2 }'
echo -n "Adobe Flash Plugin: "
curl -s -m 3 http://www.adobe.com/de/software/flash/about/ | grep -A 2 "Firefox, Mozilla, Netscape, Opera"  | tail -1 | perl -p -i -e "s/<.*?>//g," | awk '{ print $1 }'

 
For the purpose of this article, the amount of checks is limited to three. Most likely you want to keep track of versions for more than three programs. Even if one assumes that you know the versions of all programs on your computer off by heart and you always install the latest software when available it is still easy to miss if an update is available just by looking at the script output. Especially with a lot more than just three…

In this scenario it would be helpful if the check could remember the version for each program from the previous run. Again, like with the example above on how to scrape websites, there are a million ways to keep track of such data, may it be a flat file (with either shell wizardry in-place editing/comparisons or using a revision control system) or a full blown Oracle Database 11g.

As I only want to store four values (program name, current version, previous version, error count) I decided to use sqlite3. This allows for a future migration to a proper database if the need arises.

Creating a database with sqlite3:

$ sqlite3 win32.db "create table win32 (program varchar(30) PRIMARY KEY, cver TEXT, pver TEXT, err INT default '0');"

 
Now, the database needs content. With just three examples, it would be faster to just manually enter the data.

Here are the individual commands:

$ sqlite3 win32.db "insert into win32 (program, cver, pver) values ('Metapad','3,6','3,6');"
$ sqlite3 win32.db "insert into win32 (program, cver, pver) values ('Pidgin','2.9','2.9');"
$ sqlite3 win32.db "insert into win32 (program, cver, pver) values ('Adobe Flash Plugin','10.3.181.34','10.3.181.34');"

 
Checking if all data is in the database:

$ sqlite3 -header -column win32.db "select * from win32;"
program     cver        pver        err       
----------  ----------  ----------  ----------
Metapad     3,6         3,6         0         
Pidgin      2.9         2.9         0         
Adobe Flas  10.3.181.3  10.3.181.3  0  

 
As seen, the data is available. But as the check script requires the functionality to write to the sqlite3 database this can be exploited to fill the database with initial values.

At first, delete the database and recreate it from scratch:

$ rm win32.db
$ sqlite3 win32.db "create table win32 (program varchar(30) PRIMARY KEY, cver TEXT, pver TEXT, err INT default '0');"

 
For the script to write to the database, a function will do fine. The function takes the name and version as arguments, strips ^M and leading/trailing whitespaces from the version string and then writes the data to the database. The following shows the script that will fill the database with initial values.

Example:

#!/bin/bash

DATABASE="win32.db"

if [ ! -e ${DATABASE} ]
then
    echo "Error: Can not find sqlite3 Database to update/check"
    exit 10
fi
function updatedb()
{
    name=$1
    version=$2
    sver=$(echo $version | tr -d '\015' | sed 's/^[ \t]*//;s/[ \t]*$//')
    sqlite3 ${DATABASE} "insert into win32 (program, cver, pver) values ('$name','$sver','$sver');"
}

VERSION=$(curl -s -m 3 http://liquidninja.com/metapad/download.html | grep -A 5 "Latest Release"  | tail -1 | sed "s/Version //g")
updatedb "Metapad" "$VERSION"

VERSION=$(curl -s -m 3 http://developer.pidgin.im/wiki/ChangeLog | grep version | head -2 | tail -1 | perl -p -i -e "s/<.*?>//g,"  | awk '{ print $2 }' )
updatedb "Pidgin" "$VERSION"

VERSION=$(curl -s -m 3 http://www.adobe.com/de/software/flash/about/ | grep -A 2 "Firefox, Mozilla, Netscape, Opera"  | tail -1 | perl -p -i -e "s/<.*?>//g," | awk '{ print $1 }')
updatedb "Adobe Flash Plugin" "$VERSION"

Check if all the data has made it into the database as intended:

$ sqlite3 -header -column win32.db "select * from win32;"
program     cver        pver        err       
----------  ----------  ----------  ----------
Metapad     3,6         3,6         0         
Pidgin      2.9         2.9         0         
Adobe Flas  10.3.181.3  10.3.181.3  0 

 
As you can see, the “Adobe Flash Player” string looks truncated.

Better check that:

$ sqlite3 -header -column win32.db "select * from win32 where program like 'Adobe%';"
program             cver         pver         err       
------------------  -----------  -----------  ----------
Adobe Flash Plugin  10.3.181.34  10.3.181.34  0

 
Only the output was truncated, not the actual value.

As the database now has an intial state, the script requires some modifications so it can use and update the database. The function updatedb only requires a minor modification but at the end of the script there are some sqlite3 commands that check the version (and failed checks) and outputs program names where there is an update available.

This is what the complete script looks like:

#!/bin/bash

DATABASE="win32.db"

if [ ! -e ${DATABASE} ]
then
    echo "Error: Can not find sqlite3 Database to update/check"
    exit 10
fi

function updatedb()
{
    name=$1
    version=$2
    sver=$(echo $version | tr -d '\015' | sed 's/^[ \t]*//;s/[ \t]*$//')
    sqlite3 ${DATABASE} "update win32 set cver='$sver' where program='$name';"
}

VERSION=$(curl -s -m 3 http://liquidninja.com/metapad/download.html | grep -A 5 "Latest Release"  | tail -1 | sed "s/Version //g")
updatedb "Metapad" "$VERSION"

VERSION=$(curl -s -m 3 http://developer.pidgin.im/wiki/ChangeLog | grep version | head -2 | tail -1 | perl -p -i -e "s/<.*?>//g,"  | awk '{ print $2 }' )
updatedb "Pidgin" "$VERSION"

VERSION=$(curl -s -m 3 http://www.adobe.com/de/software/flash/about/ | grep -A 2 "Firefox, Mozilla, Netscape, Opera"  | tail -1 | perl -p -i -e "s/<.*?>//g," | awk '{ print $1 }')
updatedb "Adobe Flash Plugin" "$VERSION"


sqlite3                 ${DATABASE} "update win32 set err = 0 WHERE cver != '';"
sqlite3 -header -column ${DATABASE} "select program AS ProgramName,err AS ErrorRuns from win32 where err >= 5;"
sqlite3                 ${DATABASE} "update win32 set err = err  + 1 WHERE cver='';"
sqlite3 -header -column ${DATABASE} "select program AS ProgramName,pver AS OldVersion,cver AS CurrentVersion from win32 where ( cver != pver and cver != '');"
sqlite3                 ${DATABASE} "update win32 SET pver = cver WHERE ( pver != cver and cver != '' ); "

The function updatedb was modified to update the current version column and five sqlite3 commands were added.

There is a fourth field in the database called ‘err’ which is an error counter. This is used to detect if the check for a version returns an empty string and for how many consecutive executions of the program.

Set the error counter to zero when the current version contains anything but an empty string.
Display the program name and error counter if the error counter is equal or greater than five.
Increase the error counter when the current version contains an empty string.
Show the program name, the previous version and current version if current version is not an empty string.
Set previous version to the value of current version when they are not equal and current version is not an empty string.

This should catch the problem when a check returns nothing. The check may be broken and needs attention/fixing.
In any other case the version string may be some odd text which can be easily spotted when the current version and the previous version differ.

To check if the script works as intended, the pver value for one program can be modified using the following command:

$ sqlite3 win32.db "update win32 set pver = 12345 where program like 'Adobe%';"
$ sqlite3 -header -column win32.db "select * from win32;"
program     cver        pver        err       
----------  ----------  ----------  ----------
Metapad     3.6         3.6         0         
Pidgin      2.9.0       2.9.0       0         
Adobe Flas  10.3.181.3  12345       0  

 
Running the script now should provide an output:

$ ./checkscript.sh
ProgramName         OldVersion  CurrentVersion
------------------  ----------  --------------
Adobe Flash Plugin  12345       10.3.181.34   

 
Also, the OldVersion should be overwritten by the value of CurrentVersion:

$ sqlite3 -header -column win32.db "select * from win32;"
program     cver        pver        err       
----------  ----------  ----------  ----------
Metapad     3.6         3.6         0         
Pidgin      2.9.0       2.9.0       0         
Adobe Flas  10.3.181.3  10.3.181.3  0 

 
A subsequent run of the script should not yield any output, unless the vendor updated the version in the time frame we ran the script again. But does the error counter work as intended? Let’s find out!

Manually setting the current version to an empty string:

$ sqlite3 win32.db "update win32 set cver='' where program = 'Metapad';"
$ sqlite3 -header -column win32.db "select * from win32;"
program     cver        pver        err       
----------  ----------  ----------  ----------
Metapad                 3.6         0         
Pidgin      2.9.0       2.9.0       0         
Adobe Flas  10.3.181.3  10.3.181.3  0 

Rerunning the five sqlite3 commands and checking the result:

$ sqlite3                 win32.db "update win32 set err = 0 WHERE cver != '';"
$ sqlite3 -header -column win32.db "select program AS ProgramName,err AS ErrorRuns from win32 where err >= 5;"
$ sqlite3                 win32.db "update win32 set err = err  + 1 WHERE cver='';"
$ sqlite3 -header -column win32.db "select program AS ProgramName,pver AS OldVersion,cver AS CurrentVersion from win32 where ( cver != pver and cver != '');"
$ sqlite3                 win32.db "update win32 SET pver = cver WHERE ( pver != cver and cver != '' ); "
$ sqlite3 -header -column win32.db "select * from win32;"
program     cver        pver        err       
----------  ----------  ----------  ----------
Metapad                 3.6         1         
Pidgin      2.9.0       2.9.0       0         
Adobe Flas  10.3.181.3  10.3.181.3  0  

Running the five sqlite3 commands another four times will bump the error counter to 5 and it should look like this:

sqlite3 -header -column win32.db "select * from win32;"
program     cver        pver        err       
----------  ----------  ----------  ----------
Metapad                 3.6         5         
Pidgin      2.9.0       2.9.0       0         
Adobe Flas  10.3.181.3  10.3.181.3  0 

The next time the 5 commands are run and current version is still an empty string, the following should happen:

$ sqlite3 -header -column win32.db "select program AS ProgramName,err AS ErrorRuns from win32 where err >= 5;"
ProgramName  ErrorRuns 
-----------  ----------
Metapad      5         
$ sqlite3                 win32.db "update win32 set err = err  + 1 WHERE cver='';"
$ sqlite3 -header -column win32.db "select program AS ProgramName,pver AS OldVersion,cver AS CurrentVersion from win32 where ( cver != pver and cver != '');"
$ sqlite3                 win32.db "update win32 SET pver = cver WHERE ( pver != cver and cver != '' ); "
$ sqlite3 -header -column win32.db "select * from win32;"  
program     cver        pver        err       
----------  ----------  ----------  ----------
Metapad                 3.6         6         
Pidgin      2.9.0       2.9.0       0         
Adobe Flas  10.3.181.3  10.3.181.3  0      

The error counter did not get reset but the program was shown where obtaining the version has failed.
Manually setting current version again to simulate that the next run of the version check for Metapad was successful:

$ sqlite3 win32.db "update win32 set cver='3.6' where program = 'Metapad';"
$ sqlite3 -header -column win32.db "select * from win32;"
program     cver        pver        err       
----------  ----------  ----------  ----------
Metapad     3.6         3.6         6         
Pidgin      2.9.0       2.9.0       0         
Adobe Flas  10.3.181.3  10.3.181.3  0         
$ sqlite3                 win32.db "update win32 set err = 0 WHERE cver != '';"
$ sqlite3 -header -column win32.db "select program AS ProgramName,err AS ErrorRuns from win32 where err >= 5;"
$ sqlite3                 win32.db "update win32 set err = err  + 1 WHERE cver='';"
$ sqlite3 -header -column win32.db "select program AS ProgramName,pver AS OldVersion,cver AS CurrentVersion from win32 where ( cver != pver and cver != '');"
$ sqlite3                 win32.db "update win32 SET pver = cver WHERE ( pver != cver and cver != '' ); "
$ sqlite3 -header -column win32.db "select * from win32;"    
program     cver        pver        err       
----------  ----------  ----------  ----------
Metapad     3.6         3.6         0         
Pidgin      2.9.0       2.9.0       0         
Adobe Flas  10.3.181.3  10.3.181.3  0       

Now you should have a utility that can check for versions and tell you if new versions are available every time it is run.
What you can do from there is left to the imagination of the reader :)

Advertisements

One thought on “How to keep up with new versions of programs

  1. Pingback: How to query software program vendor websites for the program version using ruby « Simple Things

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s