Liv & Paul’s Web Page

I’ve updated my wishlist for Christmas, feel free to buy something not on the list though! The wishlist is here

We have moved our internet provider & phone line from Virgin Media to BT Broadband. Thankfully it has been relatively painless, especially as it also knocks out the TV. Rough summary of what needed to be done:

• Disconnect old router.
• Connect the new router to the phone line.
• Wait for router to connect to the internet.
• Connect laptop to wireless network
• Browse to the router admin menu and change:
  • Admin password
  • Subnet Address & Router address
  • Wireless password & Wireless Network Name
• Connect up all wired connections
• In router admin change:
  • Set all node names and fixed IP addresses
  • Add port forwarding for 5060, 10000-15000, 80 & 443 to server
  • Configure DynDNS
• Any other wireless devices (Macbook2, Squeezebox) update Wireless config.

A summary of the recipes that are popular with the kids. Well popular with Henry at least, May only seems content with munching through a pork pie.:

• Macaroni Cheese [H & M]
• Carbonara – Jamie’s Courgette version [H & M]
• Spaghetti Carbonara [H & M]
• Noodles with Cream [H & M]




• Shepard’s Pie [H]
• Fish Pie [H]
• Porridge [H]
• Roast Dinner Vegetables [H]
• Cauliflower/Broccoli Cheese [H]
• Fish Finger’s and Veg [H]

Henry definitely prefers mild flavours, anything too flavoursome is ‘too spicy’, from tomatoes to bacon. Or anything that cannot be chewed to a paste (meat for example, stringy vegetables).

Probably going to do a clean install of our macbook, upgrading to Mac OS 10.6.3. Need to re-install:

• Onyx – Optimisation Utility
• Mythtv – Media Front end
• Firefox & Chrome – Browsers + Adblock (Plus V 1.3.6 – easy list
• X-Tools & X Windows
• OpenOffice
• VLC – Media Player
• Picasa / iPhoto – Photo Organisers
• BBC iPlayer Desktop

Pre-server upgrade, need to upgrade MythTV, just as I got used to svn, MythtTV has moved to github. Will work out how to do patches later (see Using Git guide, for now grab a snapshot tarball:
wget --no-check-certificate https://github.com/MythTV/mythtv/tarball/v0.24

Make a clean copy. Then on the server and all clients take a copy of the latest revision in /home/mythtv/XXXXX.

Shutdown all clients and the backend using /etc/init.d/mythbackend stop, stop any cron job that auto starts the backend.

On server backup myth database as described on the MythTV Wiki using mythconverg_backup.pl.

Create the new version of MythTV: Run the previous configure recorded in config.log, updated as appropriate. Run make.

Finally shutdown all clients and the backend again. Ensure any local modifications to myth are in ~/.mythtv (i.e. mysql.txt and any menu modifications).

Remove any copies of mysql.txt (/usr/local/share/myth/mysql.txt & /var/run/mythtv/.mythtv/mysql.txt).

Usually a good idea to run mythtv-setup, on macbook start X11, run:

Restart the server using /etc/init.d/mythbackend start. Check database is upgraded okay, check detected all tuners, listings, recordings and videos. Restart the client and check it is working (live TV, recordings and videos).

Always worried that we are bringing a lot of incorrect settings and unwanted orphan database guff. We could now create a new mythconverg database using the mc.sql file. Restart the backend pointing at the new database, this will populate with working defaults. Run mythtv-setup to add tuner cards, listings provider and channels. Run mythfilldatabase to get listings. Restart client and set up appropriate Video Playback Profile, check live TV works. Import videos, music and photos. Import old database tables – previously recorded programs (oldrecorded), current recordings (recorded, recordedprogram & recordedseek), and supporting tables: (recordedrating, recordedcredits, people, credits).

Once successful, install mythweb on the server:

Git Useful links

• Using git with MythTV
• GitHub
• Git Guide
• MythTV on GitHub

Getting MythTV from Github

First install and configure git:

Grab a copy of the repository and swap to the fixes branch:

There does not seem to be a simple method of tracking what revision you have in git? Proposed upgrade process:

• Take a copy of the installed source cp -r mythtv mythtv_installed
• Review changes since this build git short-log HEAD..???
• Get the latest version of the code git pull

Time to upgrade the server again, getting numerous failures, so need to re-install the whole system…

Quick Overview of Steps to Commission the Server:

1. Burn Gentoo 2011.0 Minimal CD, and boot server using it.
2. Partition Hard Disk

Device      Boot    Start         End      Blocks   Id  System      Mount
/dev/hda1   *           1           7       56196   83  Linux       [/boot]
/dev/hda2               8          70      506047+  82  Linux swap  [swap]
/dev/hda3              71         800     5863725   83  Linux       [/]
/dev/hda4             801        9729    71722192+   5  Extended
/dev/hda5             801        1530     5863693+  83  Linux       [/mnt/portage]
/dev/hda6            1531        2260     5863693+  83  Linux       [/home]
/dev/hda7            2261        2869     4891761   83  Linux       [/mnt/root]
/dev/hda8            Remainder                                      [/shared/]

3. Follow Gentoo Installation Instructions




Use backed up files from old installation as reference to various steps in the Gentoo install guide (/etc/conf.d/*, /etc/fstab, /etc/make.conf, and so on).

Ensure kernel has support for all filesystems ext2/3, xfs, nfs server. Ensure drivers built for all key components (use lspci -v to get key ones), correct CPU is chosen, hardware sensors and CPU powersaving.

Copy across all /etc/portage files before emerging the following packages.

Once complete will have the following services running:

• apache2
• asterisk
• dahdi
• lm_sensors
• local
• mysql
• mythbackend
• net.eth0
• netmount
• nfs
• nfsmount
• ntp-client
• ntpd
• sshd
• syslog-ng
• udev-postmount
• vixie-cron
• xinetd

Use /usr/lib/portage/world as a guide as to what packages to install.

Top level services that need to be installed:

• MythTV Backend
• Apache Web Server
• Asterisk PBX
• Squeezebox Server
• MySQL Database

Key components to support the above:

• LinuxTV V4L-DVB
• DAHDI

Order of Installation:

Utilties

• syslog-ng
• vixie-cron
• gentoolkit
• logrotate
• vim
• ntpd & ntp-client
• ssmtp

Hardware Control

lm_sensors – follow installation instructions and install appropriate modules
CPU Frequency control – Use kernel modules if possible, explore /sys/devices/system/cpu/cpu0/cpufreq/

NFS Exports

Install and restore exports file from backup, TODO: Check for portmapper/rpcbind.

Mysql

1. Install the server: See the Gentoo Docs
   
   1. emerge –ask mysql
2. Update /etc/mysql/my.cnf based on previous tweaks (Mark updates with comments!)
3. Start the server:
   
   1. rc-update add mysql default
   2. /etc/init.d/mysql start
4. Import user privileges using this useful script:
   
   1. mygrants()
   2. {
   3.   mysql -B -N $@ -e "SELECT DISTINCT CONCAT(
   4.    'SHOW GRANTS FOR ''', user, '''@''', host, ''';'
   5.    ) AS query FROM mysql.user" | \
   6.   mysql $@ | \
   7.   sed 's/\(GRANT .*\)/\1;/;s/^\(Grants for .*\)/## \1 ##/;/##/{x;p;x;}'
   8. }
5. Import ‘working’ copy of various databases (asterisk, mail & mythconverg).

Asterisk

See the previous post on Asterisk upgrade.

Also install the latest phpagi

Install sox for increasing gain on voicemail.

Get British audio from Enicomms, put in /var/lib/asterisk/sounds/en_GB

Apache

Install as per Gentoo instructions, and restore backed up www files. (including .htaccess). TODO:

Download and install, and configure useful tools:

• net2ftp
• phpMyAdmin
• shell

Copy across bespoke tools:

• Asterisk Manager
• Share Research

Mythtv

Only really need backend support, but still need all the dependencies, so get portage to add them all. Ensure version we want is enabled in portage, update package.keywords

echo "media-tv/mythtv ~x86" >> package.keywords
emerge --onlydeps --ask mythtv
emerge --noreplace mythtv

Restore /root/.mythtv settings.

Also need xmltv for listings, with imdb support:

emerge --ask xmltv
tv_imdb --download --imdbdir /shared/imdb --prepStage all

Kernel Drivers if possible:

• Hauppauge WinTV-NOVA-T PCI – Ensure ‘CONFIG_VIDEO_CX88_DVB’ enabled.
• Twinhan Technology Co. Ltd VisionPlus DVB card – Ensure BT878 & BTTV enabled
• Hauppauge Nova-T USB Stick – Need DVB-USB-DIB0700 & Firmware dvb-usb-dib0700-1.20.fw

Copy backed up udev rules to ensure known adapter names. cp backups/etc/11-dvb.rules /etc/udev/rules.d/11-dvb.rules

Other Bits n Bobs

• ctorrent – For downloading torrents
• lftp – For website backups
• netkit-ftpd & xinetd – For FTP access
• dvd+rw-tools – For Backup to DVD
• git – For access to MythTV repo
• svn – For access to WordPress repo
• hdparm – For shutting down drives
• xfsutils – For xfs maintenance
• xpdf – To support research note download
• ethtool – In support of Wake-on-LAN

Weighing up the pros and cons of upgrading various components, try and capture what we have:

Motherboard –
CPU – AMD Athlon
PCI – Hauppauge Nova-T
PCI – Twinhan
PCI – Nvidia VGA
PCI – Digium Analogue Telephony Card with 1 FXO [5W max]
USB – Hauppauge Nova-T Stick
IDE – HDD
IDE – HDD
IDE – HDD
Fan – 120mm
Fan – 120mm
PSU –

New Server

ASUS P5ND2-SLI Power Consumption 150W(idle) – 220W(load) [1] Manual [pdf] – Manual states power range from 500W to 350W (system consumption) 17 to 25A on the 12V line.

Processor installed is a Pentium 4 with HT support [1], support EIST, max TDP of 85W.

With no hob or oven, need a repertoire of simple meals that we can cook in the microwave (which also has a basic grill).

Beans on Toast
Spaghetti Hoops on Toast
Fishfingers, peas & sweetcorn
Burgers
Noodles, peas & sweetcorn
Rice & Naan bread
Tinned Spaghetti Sausages
Soup
Creamy parmesan, quick cook macaroni
Chicken Pie ready meal & peas

Take away pizza / fish n chips

Pre-cooked meals:
Wraps with frozen Chilli Mince
Frozen Fish Pie

Going to try classic Delia Roasted Stuffed Goose with Prunes in Armagnac from Delia Smith's Complete Cookery Course.

I’ll be cooking:

• Roast Goose – 5lb Goose takes 1/2 hour @ 220C 3 hours @ 180C
• Apple Stuffing (in Body Cavity)
  • Bramley Apples
  • Prunes
  • Onion
  • Armagnac
  • Cloves & Mace
  • Salt & Pepper
• Forcemeat Stuffing – Neck End
  • Goose Livers
  • Cox’s Apple
  • Minced Pork/Saucemeat
  • Onion
  • Breadcrumbs
  • Dried Sage
  • Salt & Pepper
• Prunes in Armagnac
  • Prunes
  • Sugar & Armagnac
• Roast Potatoes (Maris Piper)

Plan:

• Christmas Eve
  • Prepare Prunes in Armagnac
• Christmas Day
  • 8:00am, chop Onion, Apples, Prunes – mix Apple Stuffing, tuck into Goose.
  • 8:15am, chop Goose livers, apple, onion, mix Forcemeat stuffing, tuck into Goose.
  • 8:30am, put Goose in oven @ 220C
  • 9:00am, lower oven to @180C
  • 11:00am, peel potatoes and put in pan of boiling water
  • 11:10am, bring potatoes to the boil for 10 minutes
  • 11:20am, drain potatoes, put roasting tray in oven with fat (from Goose)
  • 11:30am, put potatoes in heated roasting tray, cover and toss all pots., put in the top of the oven
  • 12:00pm, remove Goose from oven, turn up to 220C, toss pots.
  • 12:30pm, remove roast potatoes

The server is running Gentoo, which will by default only install Asterisk 1.2. Unfortunately it is not obvious that this supports the more up to date dahdi drivers required for the Openvox A400P card that I need to install.

I also could not find anything on the web to recommend a stable version of Asterisk, so I just install the latest ebuild available. To enable installation of dahdi and Asterisk with mysql cdr support I did the following:

• Save my /etc/asterisk configuration files
• Save the content of /var/lib/asterisk/agi-bin
• Update /etc/portage/package.use to enable dahdi support in Asterisk:

  net-misc/asterisk-addons mysql
  net-misc/asterisk dahdi

• Enable the installation of the latest Asterisk packages, by adding the following to /etc/portage/package.keywords

  net-misc/asterisk-addons ~x86
  net-misc/asterisk-app_notify ~x86
  net-misc/dahdi ~x86
  net-misc/asterisk ~x86
  net-misc/asterisk-core-sounds ~x86
  net-misc/asterisk-extra-sounds ~x86
  net-misc/asterisk-moh-opsound ~x86
  net-misc/dahdi-tools ~x86
  net-libs/libpri ~x86

• Remove the conflicting version of asterisk-addons:

  emerge --remove asterisk-addons

• Upgrade Asterisk and install dahdi drivers:

  emerge --ask asterisk asterisk-addons dahdi dahdi-tools

• Use etc-update to upgrade all the /etc/asterisk files, I just overwrote the old versions and manually re-inserted my config, the important files where:

  indications.conf - 'country=uk'
  modules.conf - load => cdr_addon_mysql.so
  extensions.conf - add my macros and contexts
  sip.conf - add my Outbound SIP Registrations and Providers and Devices, configure network domain

• Add dahdi to init rc-update add dahdi default
• Load the kernel modules:

  modprobe dahdi
  modprobe wctdm

• Configure dahdi:

  dahdi_genconfig
  echo "#include dahdi_channels.conf" >> /etc/asterisk/chan_dahdi.conf
  /etc/init.d/dahdi start
  /etc/init.d/asterisk restart

• Restore the content of /var/lib/asterisk/agi-bin

Finn Capital comment on ABM and HAT:

H & T – BUY [pdf]
Albemarle & Bond – BUY [pdf]

Not as painful to do as I thought. Partly because I spread the chores over a few days. On Tuesday get the bottles ready and fill with sterilising solution, Wednesday add the Finings to the brewing bin,Thursday 16th rinsed the bottles and finally bottle the beer (7 days after putting in the brew bin).

Beer is now sitting in a warm cupboard for a couple of days before going back out to the garage. Got another 30 empties in the garage, so once we are through the first 10 pints can start a new brew kit.

We’ve been in the new house for a couple of months now so have been wanting to get some beer on the go. So ordered the tried and tested “Woodforde’s Wherry”, one of Woodforde’s Norfolk Ales’ brew kits. I ordered it online this time for a good price of £15.73 from Brew It Yourself.

Done the easy bit of getting it into the brewing bin after sterlising everything, annoyingly broke my thermometer so not sure it’s at the ideal temperature for the yeast. Going to bottle this one so have got the unenviable task of sterilising 40 odd bottles in the next few days…

I quickly hacked together a WordPress plugin to help friends and family keep track of which Mr. Men books Henry has in his collection. The code is in a pretty ropey state and does not take advantage of all the new WordPress plugin API’s but thought it worth making available in case anyone wants to take it further.

Although it’s called ‘Mr. Men’ wishlist plugin it will take any list of books/CD’s available through Amazon and create a simple tiled array of covers. I grabbed the images and amazon reference numbers (ASIN) values using a simple perl script that accessed the Amazon Web Services API. However that was a while ago and the API has changed quite a bit since then.

Although I started to make it possible to have multiple wishlists but have not finished writing this bit yet.

Available to download from here: Mr. Men Plugin[zip]

May in some ways has been much quicker at picking up stuff than Henry was, walking, climbing, eating with a spoon, drawing, etc. Partly as she has a big brother to show her the way…

However her vocabulary is frustratingly minimal, which is odd as she knows she is guaranteed a smile from us when she shouts out ‘Trac-Tor’!

Here are the ones I can remember:

By By – Bye Bye
Eh Lo – Hello
Tongue Clicking – ‘Clip Clop’ = Horse
Moo – Moo (Cow, Sheep, Monkey, etc.)
Eh Oh – Uh Oh (after dropping anything)
Bum Bum – Bum Bum (Henry’s Contribution to May’s education)
Aap – Up
Trac Tor – Tractor
Ca – Cat

Bit of a nightmare getting the database schema upgraded, and I am reluctant to throw away all my recording history for the last 7 years. The problem I created for myself is that I misunderstood the wiki page detailing how to remove the database corruption, last time I was considering the upgrade. Basically a year ago I converted the database using the instructions in the guide, upgraded to the latest version of MythTV 0.21 – Fixes and assumed that was it fixed.

Alas MythTV has been inserting latin1 encoded characters into the database ever since, so I have a mixture of encoding methods in the database, options are:

1. Throw away the database, and somehow import the old recordings, or ditch all my old recordings.
2. Work out when I did the database fix and somehow convert the latter part of the database into the correct encoding method.
3. Work out when I did the database fix, ditch everything after that point in time.
4. Convert the whole database into utf8 encoding using scripts.

So lets try the one with the least amount of manual messing about first! Apparently Mysql will attempt to munge incorrectly encoded characters into the correct encoding format when you insert the data. So based on the advice found at Blue Box Group I took a new backup of the mythconverg database, copied it and changed the encoding type from latin1 to utf8. Using the sed script ‘s/DEFAULT CHARSET=latin1/DEFAULT CHARSET=utf8/‘.

This did not appear to convert the characters, tried lots of combinations to try and convert the database with no great progress. Examining the hex dump of some of the UTF sequences did not seem to make any sense, so I’m taking the view that the data is corrupt.

Next attempt involves simply stripping out all the non-ASCII characters and replacing with ‘?’ characters. Easy to do in vi with something like ‘s/[^ -~][^ -~]*/?/g‘.

This will generate a few problems in that the INDEX’s may now exceed the maximum length, so wherever ‘hostname’ is used as a key in the index change it to `hostname`(20). Additionally there will almost certainly be duplicates in the people table, so I removed the line UNIQUE KEY `name` (`name`(41)) where the people table is CREATED. Had to also change the KEY to the oldprogram table to be PRIMARY KEY (`oldtitle`, `airdate`).

Create a new mythconverg23 database with appropriate user privileges and import the modified database. I wrote a short php script to find and remove all the duplicate people from the people table and to remap the credits entries.

To check this was successful recreate the index on the people table with the sql command: ALTER TABLE `people` ADD UNIQUE (`name` ( 41 ));.

Export the database again, change the encoding back to latin1, and import back into mysql.

mysqldump -u mythtv -pmythtv --opt --skip-set-charset \
  --default-character-set=utf8 --skip-extended-insert \
  mythconverg23  > mythconverg23_utf8.sql
cat mythconverg23_utf8.sql | \
sed 's/DEFAULT CHARSET=utf8/DEFAULT CHARSET=latin1/ \
      s/`hostname`(20)/`hostname`/' > mythconverg23_latin.sql
cat mythconverg23_latin.sql | mysql -u mythtv -pmythtv mythconverg23

Okay see if we can now upgrade the schema…

mythbackend -v all