... In Which It Is Revealed How An AHCI Bug Makes One's Insyde(s) Freeze

I found this code in Intel's AHCI Option ROM from the Insyde BIOS. It appears to be code to build the Translated Device Parameter Table (which is a slightly different¹ implementation than the one documented in the Enhanced Working T13 Draft 1126DT).

Psuedo code  (version: Serial ATA AHCI BIOS, Version iSrc 1.20_E.0019 07092009):

Function Create TDPT for drive
    Read partition table with INT13 0x201
    If read fails, or 0xAA55 signature isn't present, goto Calculate
    Get head,sectors from FIRST Partition table entry, Ending CHS values
    heads = head+1  (because of 255 limit in partition table)
    For each partition entry
        If BOOTABLE (entry[0] == 0x80) goto UsePartition
    EndFor
    For each partition entry
        if (entry[0] == 0) and (entry[4] != 0) goto UsePartition
    EndFor


Calculate
    Call CalculateCHS - using DPT and physical size
    if no need for translation, return GOOD
    Goto CreateTDPT with cylinders, heads, sectors


UsePartition:
    Read first sector of partition with INT13 0x4200
    If the word at offset 0x1A is less than 0x100,
      and the word at offset 0x18 is less than 0x40
    then
        set heads to the byte at offset 0x1A
        set sectors to the byte at offset 0x18
    fi
    tracksize = heads * sectors
    if tracksize == 0, Goto Calculate
    DWORD size = (DPT[heads]*DPT[sectors])*DPT[cylinders]
    WORD cylinders = size / tracksize <--- Bad!
    -- if the result is greater than 65536, a divide overflow occurs

    -- which isn't handled by the BIOSes

    if (cylinders > 1024) cylinders = 1024
    if ((heads == DPT[heads]) && (sectors == DPT[sectors])) return GOOD


CreateTDPT:
    WORD at DPT[8]  = DPT[0] - Save original cylinders
    BYTE at DPT[10] = DPT[2] - Save original heads
    BYTE at DPT[7]  = DPT[3] - Save original sectors
    WORD at DPT[0]  = cylinders
    BYTE at DPT[2]  = heads
    BYTE at DPT[3]  = sectors
    BYTE at DPT[5]  = 8 if heads greater than 8, otherwise 0
    BYTE at DPT[4]  = 0xA0
    BYTE at DPT[15] = SUM( DPT[0] .. DPT[14] )

So, what goes wrong? When it breaks, it starts with bad values from the partition table, and tries to fix it with values from the boot parameter block, if it finds "valid" numbers there for heads and sectors (that is, less than or equal to 0xFF and 0x3F, respectively). When these values aren't right,  due to full disk encryption, an operating system other than Microsoft Windows, or malicious intent:

• It uses the ending head/sector of the first partition to size the translation layer.
• Windows 7 with 100MB partition results in unexpected values for INT13, FUNCTION=8 (eg, 0x13 heads).
• It stores those values into the Translated Device Parameter Table..  and then some other code comes along and uses those values. While I can't find where those values are causing the exception, anything doing C/H/S translation will be unhappy.

Looking back at version Serial ATA AHCI BIOS, Version iSrc 1.20E (Gigabyte Desktop Motherboard), I found that it doesn't read from the BPB at all.  I speculate the extra read of the NTFS boot-sector was to workaround a problem on Insyde BIOS.   This version can be crashed if the two bytes in the partition table are small enough and will hang with error code 23. Award BIOS will function OK with the other unexpected values, but Insyde BIOS will still crash if it sees them.

Finally, one HP system with an Insyde BIOS has the latest(?) 'fixed' version (Serial ATA AHCI BIOS, Version iSrc 1.20_E.0024 12212009), which reads from both the partition table and the BPB, also adding  still more checks. Unfortunately, it seems as though someone messed up and added a further bug, as it doesn't actually use any of the values it reads, but rather discards them all.

New and improved UsePartition (Serial ATA AHCI BIOS, Version iSrc 1.20_E.0024 12212009):

Read first sector of partition with INT13 0x4200

if the word at offset 0x1FE is not equal 0xAA55

   and the byte at offset 0 is not equal 0xEB

   and the word at offset 0x1A is less than 0x100

then

    set heads to the byte at offset 0x1A

fi

if (tracks == 0) or ((sectors & 0x3F) == 0) Goto Calculate

-- New bug: Since sectors can be at most 0x3F from partition table

-- the newer version ALWAYS goes off to Calculate the CHS

if (sectors & 0xC0) == 0)  Goto Calculate

tracksize = heads * sectors

if tracksize == 0, Goto Calculate

DWORD size = (DPT[heads]*DPT[sectors])*DPT[cylinders]

WORD cylinders = size / tracksize <-- Uber dangerous

-- if the result is greater than 65536, a divide overflow occurs

-- which isn't handled by the BIOSes.

if (cylinders > 1024) cylinders = 1024

if ((heads == DPT[heads]) && (sectors == DPT[sectors])) return GOOD

Goto CreateTDPT

A year later and neither Acer nor Gigabyte are providing fixed BIOSes.

¹Expected Final TDPT Values from a 60GB SSD:

          WORD Logical Cylinders   0x400

    BYTE Heads               0xFF

    BYTE Sectors             0x3F

    BYTE Signature           0xA0

    BYTE HeadsAbove8Flag     0x08

    BYTE Ignored             0x00

    BYTE Physical Sectors    0x3F

    WORD Physical Cylinders  0x3FFF

    BYTE Physical Heads      0x10

    BYTE Ignored[4]          0x0

    BYTE Checksum            0x89

Acer 5810tz's "Secret" (Shhhhh!) BIOS Menu

As described in http://marcansoft.com/blog/2009/06/enabling-intel-vt-on-the-aspire-8930g/ and on NotebookReview's Forum, there are hidden menus in Insyde's Acer  BIOS.  Some folks modify their BIOSes to enable them; however,  on the Acer 5810, it's possible to enable some of these menus with just a setup change, which is, of course, safer than having to flash your whole BIOS just to modify a single boolean.

To do so, you need:

•  a DOS-bootable USB stick
• the flashit.exe binary from the BIOS update
• grdb from http://ladsoft.tripod.com/grdb.htm

Copy the utilities to the usb disk and create a batchfile (ex. modify.bat) and add the following:

flashit Setup A04A27F4-DF00-4D42-B552-39511302113D /rb:setup

grdb setup

flashit Setup A04A27F4-DF00-4D42-B552-39511302113D /wb:setup

     

Reboot to DOS on the USB stick, run the batchfile and in GRDB type:
e 31a 1 

w 

q

When you reboot, you should have an extra menu "Intel" below "D2D Recovery"

The "hidden setup" is usually located immediately after the "D2D recovery" variable.  If you wish to locate it on a different version of Acer Insyde bios, you can write out the setup variable ( the /RB command-line) with and without D2D recovery enabled.  The location one past that will be what you need. For example:

fc /b setup1 setup2

Comparing files SETUP1 and SETUP2

00000219: 00 01

Which is 21A (or 31a if you are using GRDB, because GRDB pretends it's a .COM file )

Although its unlikely that messing up the setup variables will cause the laptop to not boot, prudence would be to know how to use the FN-ESC "Crisis Recovery mode" and know what the filename is for your particular laptop.  (Mine is JM41X64.fd)

Possibly Useful Reference:

Post On Lenovo Forum With FlashIt Parameters

Pictures:

"Secret" menu available. 

Intel->

Intel->Power

Intel->Advanced

Intel->Advanced->Boot

The Byte That Bit Me Insyde

Since Insyde doesn't seem interested in patching its BIOS, I thought I'd share a neat way to make a laptop with an Insyde BIOS hang on boot by changing a single byte.

Caution: Back up any important data, if you're silly enough to do this on a live system; while this shouldn't mangle any of your bits, it's certainly possible it could.

Also, if you're not terribly fond of grasping naked sectors and pushing values into unfilled gaps, you might want to bail out now.

The Prerequisites:

• Laptop with buggy Insyde EFI BIOS.
• SSD or HDD (physical medium unimportant)
• Two or more partitions (which is the default Windows 7 configuration, and tends to be the default Linux installation as well).
• The SATA port configured as AHCI in BIOS.

My hardware configuration:
Acer 5810tz Timeline Notebook, with InsydeBIOS Release 1.35.

Now, with your personal favorite disk editor of choice (I'm using the very excellent, free and portable HxD):

HxD with Physical Disk 1

The Byte To Change

• Select the first partition of your first physical disk (labelled Physical Drive 1 in HxD).
• Go to sector 2048.    
• Go to offset 0x19 (25 decimal) and change from 0 to any value.
• Reboot
• Watch your laptop with an Insyde BIOS freeze.

Don't panic!  Simply:

•  disconnect the laptop drive,
•  bop into BIOS (F2)
•  change the SATA mode from AHCI to IDE.
•  Reconnect your hard drive, and boot.  You'll hit the usual Windows BSOD complaining about you trying to boot with the wrong set of disk drivers; boot into the 32 bit recovery console (since many utilities don't yet have 64bit equivalents and the WOW64 subsystem probably won't be available (especially if you're booting a mini-xp environment off a thumb drive instead))
•  Run your sector editor, and change byte at offset 0x19 back to 0.
•  Reboot normally.

So, what's going on here?

Sector 2048's where Windows 7 puts the start of its first partition.  From the partition boot sector layout we can see the particular byte is part of the boot parameter block, specifically the high byte of the Sectors Per Track word, which happens to be ignored by Windows 7.

What's a BIOS doing, caring about this?

Near as I can tell, this could be an attempt at an AHCI optimization, and the BIOS code simply fails to do a sanity check on the range.

Q. Except for the occasional black hat looking for a chuckle, who'd want to hang their laptop?

A. Anyone installing Linux, BSD, or full disk encryption such as Truecrypt and PGPwde.

These, as part of their normal operation, can change that special byte, giving much excitement and hair pulling to the lucky person whose BIOS, in an effort to be a most helpful puppy, manages to decorate the newspaper with excrement after having a chew with it.

I'm so glad EFI has led us away from incompabitle, buggy BIOSes!

Tools mentioned:

HxD Hex Editor (used for sector editing)

References:
Homepage of Insyde

Others probably running into this specific issue:
Seen on HPs, blamed on encryption
And Lenovos, blamed on Truecrypt.
Seen again, attributed to AHCI+Truecrypt conflict.
Even with BSD!

Facebook: Can't See MySpace In The Tar Pit.

Social networks can be amusing.  I have a Facebook account, and as of this blog entry, 7 friends.  Actual friends, that I either know irl, or have known virtually for over a decade... all good folks.  The posts on my wall tend to be either interesting links, or random pictures, or other interesting sort of things.  I can post links to my wall that I find interesting, to share.  It's all quite ordinary, really, and not much worth commenting on.

Except, of course, on August 28, 2010, when I went to log in, and wasn't able to, instead being greeted with this:

Confirm Your Identity
To continue, we need you to provide your phone number. This quick security check helps keep Facebook a community of real people who connect and share using their real identities.


Confirm your identity by adding a mobile number to your account

The only option on this screen was "Add Your Phone". ... no 'Cancel' or any option for another authentication method.

Since I live far beyond the range of any cell tower, and don't own a mobile phone, using one to confirm my identity with poses something of a problem; especially if I'm not of a mind to drive almost an hour and buy one just to use Facebook!  

However, I do have a Google Voice account, which can accept SMS messages (or, at least, so I thought... turns out it's not compatible with Facebook), so I went through the Add Phone, chose Provider Not Listed, entered the number, and... waited.  And waited.  And Google'd, and found a FAQ stating Google Voice couldn't accept Facebook SMS messages.

Long story short, I found myself in a loop; going to Add Phone no longer allowed entering a number, but just allowed choosing a provider.  I'd get a popup where I could select "Never Got Your Confirmation", which would redirect to a login dialog, which upon log in, would go back to the "Confirm Your Identity" message.

Somewhat bemused and more than slightly annoyed, I searched through Facebook's help forum; there weren't any similar cases. I did, however, find a strange web forum, where others were having the same issue as me, starting at the same time. It seems that Facebook decided folks without a mobile phone number really needed to provide one (data mining, anyone?) and flagged accounts without one listed in need of identity confirming. 

(As a tangent, the forum, linked at the end of the email, is the first I've stumbled across where people with the exact same issue are demanded to create one thread per person, resulting in about 50 different threads with people saying they were bumping into this issue, and if anyone replied in any of those threads that they, too, were having this issue, the moderator went into hysterics, telling them to create their own thread... very odd!)

Today, after around 13 bug submissions to various Facebook emails, a support staffer, without mentioning what the problem was, told me to try logging in again, at which time I was able to select "Use a different verification method" (which had appeared a couple days ago, and simply looped back to the Confirm Your Identity screen) and enter my Google Voice number... which then got an automated message with a PIN, that let me confirm that I actually exist.  Well, as much as any of us do, here in Universe v. ???. 

Since my Facebook usage is so scarce, my only real concern was someone sending me a message and feeling as though I was ignoring them when I didn't respond; I wasn't terribly put out about not being able to log in.  

Still, such an obvious bug existing for almost two weeks, affecting multiple users, and with incredibly half-arsed code on the public-facing end of a large website, speaks volumes about Facebook's view of their 'customers' (read: eyeballs on ads).  

It appears Facebook didn't learn from the happily sinking fad that was MySpace: people'll switch the second there's something that sucks just slightly less than your service.  Facebook isn't as God-awful terrible as Myspace, but it's working on it (want to feed my llama? C'mon!).  

The good news, and the non-ten-ton-heavy-thing light at the end of the tunnel, seems to be Diaspora: social networking 'done right', as it were.  Obviously, the attention-whores who use Facebook to try to draw as much attention as they possibly can to their existence ("Hiccup, I'm drinking, and I don't know why I'm sharing that, except I want everyone to ask me why I'm drinking right now, and what I'm drinking, because NOTICE ME, NOTICE ME, NOTICE ME!! Over here!! Look at me!!!) aren't going to be moving any time soon, but the REST of us (is there any irony when someone typing a blog accuses everyone else of being attention whores? :-) ) likely will jump ship as soon as Diaspora's usable.

To clarify, that'd be the people who share links, and their thoughts, and pictures of their dogs and sunsets, and all the sort of stuff that connects us, but not the people who share the approximate growth of their third toe's toenail in the last 5 minutes and 20 seconds.  Interestingly enough, Facebook's unlikely to muck about with the accounts of people in the latter category any time soon: Not, of course, that it matters.  The tar pit's deep, and there's plenty of room there towards the bottom.

References:

Facebook Forum - Forum not affiliated with Facebook with moderators giving to throwing hissy fits unless you create lots and lots of threads. (There was a forum created JUST for this issue "Disabled Accounts (Mobile Verification)"

Diaspora - Social Networking site that might not completely suck: open source, allows control over one's content and privacy.

Splitting a single flac file into mp3s.

I might need to stop avoiding .flac files, as now I found how easy it is to do something with them; historically, I've just been lazy and ripped my CDs to mp3s.

However, it might be prudent for me to start converting to flac, and have lossless audio I can just extract the specific tracks I want for a given rotation.

Split2flac - A beautiful shell script

References:
https://bbs.archlinux.org/viewtopic.php?id=90278
http://stillstup.blogspot.com/2008/07/split-lossless-audio-ape-flac-wv-wav-by.html

Linux VMWare BIOS Extraction

More of a note to self than a blog...

objcopy /usr/lib/vmware/bin/vmware-vmx -O binary -j bios440 --set-section-flags bios440=a b.z
perl -e 'use Compress::Zlib; my $v; read STDIN, $v, 217218; $v = uncompress($v); print $v;' < b.z > bios440_mod.rom

cp oembios440.rom /srv/projects/vm/nýttkjöt/
echo "bios440.filename = \"oembios440.rom\"" >> /srv/projects/vm/nýttkjöt/nýjar_vm.vmx

Reference:
VMware BIOS modification - for Linux users
Modifying The VMware BIOS
Persistent BIOS Infection

Exporting Email From Myspace With Google Gears in Firefox on Linux

Myspace doesn't appear to have a way of natively allowing exporting messages.  What follows is a proof-of-concept kludged script to do so.

I desired to retrieve 50 email messages from my now defunct Myspace inbox.

I didn't have any messages in sub folders, nor did I care for the single message I had in my 'sent' folder.

For whatever reason, it doesn't appear that Sent messages are being saved in the Gears database, though it ought parse those in sub folders.

I make no effort to parse and scrape user images; however, doing so appears as though it would be trivial.

Of further note: for a user that has an account number of 12345678, a user id of jdoe, and a display name as "Johnny D0e LOLZ!!!!", Myspace appears to store another display name ("John Doe") that it shows under a user's picture in email; this name is not accessible via the Gears database, as far as I can tell.  (The example above is based on an actual Myspace 'friend')

Regarding Myspace's formatting, I found the Firefox add-on "SQLite Manager" exceptionally useful for advancing my understanding of how Myspace was saving the messages; just using sqlite's .dump function didn't make things like rowid clear for a sql-language amateur such as myself;  anyone attempting to perhaps pursue the creation of a real parser would likely benefit from the use of sqlite manager.

Please note, the CreatedData appears to have milliseconds tacked on; hence the weirdness I do with trimming.  My timezone is MDT; please adjust the +/- for hour accordingly.

This all is done in Firefox on Xubuntu, as Google Gears was not available for Google Chrome on Linux.

Steps:

1. Install Google Gears: Google Gears
2. Log in to Myspace and go to your inbox; obviously, give Gears permission to sync.  Wait until messages are saved to hdd.
3. Created a directory under your home directory, perhaps "myspace_emails" or similar would be sensible.
4. With your editor of choice (or cat >> scrapemyspace.sh for the lazy/efficient) create file and copy/paste my script below into it. Give it +x.
5. Now simply run script (sh name_you_used.sh); if you wish to have all emails in ONE file, in addition to the separate ones created, do: script.sh >> allmyspaceemails.txt or whatever; this was what I needed, and why I used tee.  The dump to the sqlite3 file is for debugging, and contains the sql commands to rebuild the database (for example, in mysql...). Comment out if not required.
6. Done!

If the location where your Gears data is stored changes, simply change the path variables at the top; also useful if you want to test, by copying the local database to your home directory.

The *.default may need to be changed (~/.mozilla/firefox/profiles.ini) for multiple users.

There is NO error checking in my script: if this makes you uneasy, please feel free to add some.

You may use this script however you wish, as long as you give me attribution, but the following disclaimer also applies:

This script is provided "AS IS", without warranty of any kind, stated or implied, including without limitation the warranties of merchantability, fitness for a particular purpose and non-infringement. The entire risk as to the quality and performance of this script is borne by you. Should the script prove defective, you and not me assume the entire cost of any service and repair. etc. etc. (If you don't understand what this script does, please find someone who does, and ask them if it's okay for you to run it, instead of trusting a random person on the intrawebs.)

Final disclaimer: I know less about bash scripting than I do about SQL. Beware.

That said, enjoy!

# Script to parse Google Gear sqlite3 db with Myspace Emails

# Copyright (C) 2010, Seth Stahlman. All rights reserved

#!/bin/bash

dbpath=~/.mozilla/firefox/*.default/Google*/messaging.myspace.com/http_80/

dbfile='myspace.messaging.database\#database'

db="${dbpath}${dbfile}"

cut='cut -d, -s'
sqlite3 $db ".dump" >> myspace_email.sqlite3.sql

authors=`sqlite3 -csv $db "select rowid, * from AuthorData_content;"`

echo "$authors" |

while read a

do

        uid=`echo $a | $cut -f1`

        dn=`echo $a | $cut -f2`

        un=`echo $a | $cut -f3`

        echo "Messages from $dn ($un) (Account ID: $uid)\n" | tee -a "${uid}_${un}"

        msgids=`sqlite3 -list -separator , $db "select MessageId, \

                datetime(substr(CreatedDate, 1, 10), 'unixepoch', '+1 hour') as date from MessageMetaData where AuthorId = $uid;"`

        echo "$msgids" |

        while read m

        do

                messageid=`echo $m | $cut -f1`

                messagedate=`echo $m | $cut -f2`

                subject=`sqlite3 -list $db "select c0Subject from MessageData_Content where rowid = $messageid;"`

                body=`sqlite3 -list $db "select c1Body from MessageData_Content where rowid = $messageid;"`

                echo "Message ID: $messageid\nDate: $messagedate\nSubject: $subject\n\n$body\n" | tee -a "${uid}_${un}"

        done

done

Lookouts, Communications, Escape Routes, Safety Zones (LCES)

Today was our S130/190 refresher, and then the S134 class.

I think many of the watch out conditions in the IRPG must have been bought with a few deaths; at least, the movie we watched seems to indicate it: "3 of the men survived by reaching the ridge, but the rest..." and "It could have been avoided, if only..."

Naturally, half the examples were old Colorado fires. Nothing like having a little pride in your state...

The most fun part of the class, of course, was where we got to do the scenario as the Feds.

"Helicopter? Suuuuure, we'll take it! Let the taxpayers pay for it! Give us a slurry, too! Oh, we've got a bulldozer coming? Send it to the north side! Wait, wait, ... what do you mean our update says we've got strong SW winds now?! How the hell did that happen?! The local department needs radios? Sure, give them some; we're USFS, by golly! Are we not merciful? Hold on, hold on, what do you mean the local department's fireline's not .... wait, wait, WHAT? Why is their tender moving over there?! The IC didn't... HEY! Who the hell came up with this scenario?!"

Add some six sided dice, a dragon, and some confiscated controlled substances, and you've got a game everyone can play!

I was sort of lost throughout most the rest of the class, though... I fell asleep somewhere around L.A.D.D.E.R... (advanced listening... teehee!) Deployment zones are safety zones, right?  And you're supposed to stay out of the black? Just kidding...

The movie we watched did have some trippy rainbow colors for the safety zone part though; the convenience of mushrooms, without the trudging.

Speaking of the trudging. If 7+ hours of rapid classroom exercises and dungeon master-esque fire games wasn't just the most fun, the pack test at the end certainly was.

As I'm overweight, out of shape, and, best of all, an ex-smoker, the last time I walked 3 miles was... well, actually, I'm not sure I ever have.  Going for an arduous rating would seem to be, ahem, "ill advised". Or, for the coarser description: batshit insane.

Granted, for CTSP, fitness is 'None Required', but where's the fun in that?

For those not familiar with the criteria for the various levels of performance, they are:

• Arduous Pack Test - 3 mile hike with 45lb pack in 45 minutes.
• Moderate Field Test - 2 mile hike with 25lb pack in 30 minutes.
• Light Walk Test - 1 mile hike with no pack in 16 minutes.

My pack was 47lbs - tire chains and water jugs, plus fire shelter.  It bounced. A lot.  Interestingly enough, about halfway through, the rather brisk 35ºF day felt glorious.

My time was 43:56.

This, despite having only a passing acquaintance with exercise, way too many pounds, and a complete lack of anything resembling lung capacity.

My secret?

In Flames.

It's quite astounding, really, how one can do death metal growls with only labored exhalations.

Not in tune, of course, but in the key of "I'd hurl if I wasn't too exhausted to expel the contents of my stomach".

I told myself, "It'll all be worth it, too, the next time I flash my red card and they look at me, and back at the (A), and wonder, 'How in the hell did this guy ever pass an arduous pack test?!'"