Support extraction of El Torito HD image content

[pbatard]

Such as the one from DOSUSB ISO, as well a regular DOS ISO data.
Basically, this would copy what are expected to be DOS files to a DOS bootable USB.
We may also want to handle pure image executables such as the ones used on Memtest86+ ISOs, through Syslinux's image load or something...

Might also be able to use something similar to this script:

geteltorito.pl

#!/usr/bin/perl

use Getopt::Std;

#
# geteltorito.pl: a bootimage extractor
# Script that will extract the first El Torito bootimage from a
# bootable CD image
# R. Krienke 08/2001
# krienke@uni-koblenz.de
# License: GPL
#
# Get latest version from:
# http://userpages.uni-koblenz.de/~krienke/ftp/noarch/geteltorito
#
$utilVersion="0.6"; 
#
# Version 0.6
#    2015/02/25
#    I included a patch by Daniel Kekez, daniel.kekez@utoronto.ca to make geteltorito
#    better compatible with windows:
#       "To run this Perl script using using Strawberry Perl under Windows, I
#       found it was necessary to explicitly open the files in binary mode since
#       Windows will default to text mode when open() is called."
# Version 0.5
#    2009/06/22
#    A patch for harddisk emulation images from <colimit@gmail.com>.
#    For BootMediaType=4 (harddisk emulation) SectorCount is always 1, and geteltorito.pl
#    returns just MBR. This patch guesses the correct bootimage size
#    from MBR (offset+size of the first partitition).
# Version 0.4
#    2007/02/01
#    A patch from Santiago Garcia <manty@debian.org> to use a virtual sector
#    size (vSecSize) of 512 bytes, as defined on "El Torito" specs and change
#    unpack of the sector count from n to v to get the correct sector count.
# Version 0.3
#    2006/02/21
#    A patch from  Ben Collins <bcollins@ubuntu.com> to make the 
#    utility work on PPC machines (change from 'L'-encoding in pack to 'V')
# Version 0.2
#    Several patches included from Nathan Stratton Treadway(nathant@ontko.com)
#    to adjust the platform output as well as fixes for other minor bugs
# Version 0.1
#    Initial release
#
# For information on El Torito see 
# http://en.wikipedia.org/wiki/El_torito

$vSecSize=512;
$secSize=2048;
$ret=undef;$version=undef;$opt_h=undef;$loadSegment=undef;$systemType=undef;

#
# Read a particular sector from a file
# sector counting starts at 0, not 1
#
sub getSector{
   my ($secNum, $secCount, $file)=@_;
   my ($sec, $count);

   open(FILE, "<:raw", $file) || die "Cannot open \"$file\" \n";

   seek(FILE, $secNum*$secSize, 0);
   $count=read(FILE, $sec, $vSecSize*$secCount, 0) ;
   if( $count != $vSecSize*$secCount ){
       warn "Error reading from file \"$file\"\n";
   }
   close(FILE);

   return($sec);
}


#
# Write eltorito data into a file
#
sub writeOutputFile{
   my($name)=shift;
   my($value)=shift;

   open(OUT, ">:raw", $name)|| die "$0: Cannot open outputfile \"$name\" for writing. Stop.";
   print OUT $value;
   close(OUT);
}


#
# Usage
#
sub usage{
	warn "\n$0 [-hv] [-o outputfilename] cd-image \n",
	    "Script will try to extract an El Torito image from a \n",
	    "bootable CD (or cd-image) given by <cd-image> and write \n",
	    "the data extracted to STDOUT or to a file.\n",
	    "   -h:        This help. \n",
	    "   -v:        Print version of script and exit.\n",
	    "   -o <file>: Write extracted data to file <file> instead of STDOUT.\n",
	    "\n\n";
	exit 0;    
}


# ---------------------------------------------------------------------
$ret=getopts('hvo:');
 
if( defined($opt_v) ){
 	warn "Version: $utilVersion \n";
	exit 0;
}

if( defined($opt_h) || $#ARGV <0 ){
         usage(0);
}	 

if( defined($opt_o) ){
   $outputFilename="$opt_o";
}
	 
$imageFile=$ARGV[0];

if( ! -r $imageFile ){
	die "Cannot read image/device \"$imageFile\". Aborting\n";
}

#
# Read Sector 17 from CD which should contain a Boot Record Volume
# descriptor. This descriptor contains at its start the text ($isoIdent)
# CD001     and ($toritoSpec)
# EL TORITO SPECIFICATION
# see http://www.cdpage.com/Compact_Disc_Variations/eltoritoi.html
# for details
#

$sector=getSector(17, 1, $imageFile );
($boot, $isoIdent, $version, $toritoSpec,
 	$unUsed, $bootP)= unpack( "Ca5CA32A32V", $sector );

if( $isoIdent ne "CD001" || $toritoSpec ne "EL TORITO SPECIFICATION" ){
	die "This data image does not seem to be a bootable CD-image\n";
}	

#
# Now fetch the sector of the booting catalog 
#
$sector=getSector($bootP, 1, $imageFile );

print STDERR "Booting catalog starts at sector: $bootP \n";

# The first 32 bytes of this sector contains the validation entry for a
# boot. The first byte has to be 01, the next byte determines the
# architecture the image is designed for, where 00 is i86, 01 is PowerPC
# and 02 is Mac. More data give info about manufacturer, etc.  The 
# final two bytes must contain 0x55 and 0xAA respectively (as 
# defined by the El Torito standard).

$validateEntry=substr($sector, 0, 32);

($header, $platform, $unUsed, $manufact, $unUsed, $five, $aa)=
               unpack( "CCvA24vCC", $validateEntry);

if( $header != 1 || $five != 0x55 || $aa != 0xaa ){
	die "Invalid Validation Entry on image \n";
}

print STDERR "Manufacturer of CD: $manufact\n";
print STDERR "Image architecture: ";
print STDERR "x86" if( $platform == 0 );
print STDERR "PowerPC" if( $platform == 1 );
print STDERR "Mac" if( $platform == 2 );
print STDERR "unknown ($platform)" if( $platform > 2 );
print STDERR "\n";

#
# Now we examine the initial/defaultentry which follows the validate
# entry and has a size of 32 bytes. 

$initialEntry=substr($sector, 32, 32);

($boot, $media, $loadSegment, $systemType, $unUsed, 
       $sCount, $imgStart, $unUsed)=unpack( "CCvCCvVC", $initialEntry);

if( $boot != 0x88 ){
	die "Boot indicator in Initial/Default-Entry is not 0x88. CD is not bootable. \n";
}    

print STDERR "Boot media type is: ";
if( $media == 0 ){
	print STDERR "no emulation";
	$count=0;
}
if( $media == 1 ){
	print STDERR "1.2meg floppy";
        $count=1200*1024/$vSecSize;  
}
if( $media == 2 ){
	print STDERR "1.44meg floppy";
        $count=1440*1024/$vSecSize;  
}
if( $media == 3 ){
	print STDERR "2.88meg floppy";
        $count=2880*1024/$vSecSize;  
}
if( $media == 4 ){
	print STDERR "harddisk";
	$MBR=getSector($imgStart, 1, $imageFile );
	$partition1=substr($MBR, 446, 16);
	($unUsed, $firstSector, $partitionSize) = unpack( "A8VV", $partition1);
	$count=$firstSector + $partitionSize;
}
print STDERR "\n";

# Only use the internal sector counter if the real size is unknown
# ($count==0)
$cnt=$count==0?$sCount:$count;

print STDERR "El Torito image starts at sector $imgStart and has $cnt sector(s) of $vSecSize Bytes\n";

# We are there:
# Now read the bootimage to stdout
$image=getSector($imgStart, $cnt, $imageFile);

if( length($outputFilename) ){
   writeOutputFile($outputFilename, $image);
   print STDERR "\nImage has been written to file \"$outputFilename\".\n";
}else{
   print "$image";
   print STDERR "Image has been written to stdout ....\n"; 
}

Might also want to look at the EFI Lenovo BIOS update images, which 7-zip has trouble to extract, such as Lenovo BIOS Update n1eur34w.iso...

[saivert]

Rufus also presently does not extract all contents from EFI.img (El-Torito) to the resulting USB flash drive which means bootloaders like Systemd-boot (formerly gummiboot) wont work as the loader entries are missing. When UEFI booting in HDD mode the EFI.img isn't used at all and there is no way the bootloader would be aware of it. Essentially these .ISO images have to be written in DD mode (ISOHybrid) as that is the only way they are going to work.

[pbatard]

The current handling of efi.img is a workaround for Debian Live, so it only applies to these specific ISOs (which only need the UEFI bootloader extracted to work). I strongly suggest that other distros should try to follow the Debian example and avoid shoving more than the bootloader into the .img, because it creates problems downstream for users who prefer writing their USB media in a manner in which the content will be fully accessible from Windows.

Still, if you can provide the URL of an ISO that highlights the issue you are describing, I'll see what I can do...

[pbatard]

Note that Rufus 3.6 will introduce full extraction of efi.img files. This means that distros like Solus will boot properly, even if the UFD has been created in ISO mode.

[ryanturcotte]

Suscribed to this, I'm curious if your note would support the burning of the Lenovo .iso files they provide for BIOS upgrades.

https://support.lenovo.com/us/en/downloads/ds105487

When you try these now you just get an error "Failed to scan image"

[pbatard]

@ryanturcotte, you might be interested with this entry from the FAQ.

Looking at the content of the Lenovo ISOs, what you need to do is create a DOS drive (which you can do in Rufus), then extract the relevant content from the ISO onto it (BIOS data + DOS flashing application) and run the relevant command, which you should be able to pick from AUTOEXEC.BAT.

[sschuberth]

@pbatard, your above instructions seem to involve steps that you cannot perform with Rufus alone, right? It would be nice if Rufus included the functionality from geteltorito so that Lenovo BIOS ISOs can be directly written to USB sticks.

Edit: In my case, I'm specifically talking about the ISO at https://download.lenovo.com/pccbbs/mobiles/r0zuj09wd.iso.

[pbatard]

It would be nice if Rufus included the functionality from geteltorito

That's the whole point of this Enhancement Request.

However, I have no idea if/when I'll ever get around to add this feature, on account that the number of people this would actually benefit is small, and therefore this is a low priority enhancement. Hence the reason why it's been open for about 8 years now...

[sschuberth]

I'm actually not so sure if the number of effected users really is that small. After all, Lenovo is on of the biggest PC manufacturers and notebooks usually don't have optical drives these days, so the use of bootable USB sticks to flash BIOSes should be rather common. Maybe it's a silent majority, or the affected end-users simply are not developer-ish enough to upvote issues on GitHub... but I freely admit that's just speculation.

[pbatard]

I'm actually not so sure if the number of effected users really is that small.

I'm sure. I've seen the number of requests I've got for El Torito extraction compared to other features, and it is small.

At any rate, I have to prioritize the features I can work on, and while I understand that each user wishing for a specific one would like to see it make the top of the list, there's just no way I can make everybody happy, and I'm afraid that no amount of "But I want it now!" coming from a single user is enough to make me reconsider, especially as the Rufus FAQ has a rather solid workaround for this feature. You also have to realize that the feature you'll get when/if I ever add it will be far from the magic bullet you expect it to be (i.e. Rufus doing all the setup, and the user just having to press a button for everything to work)... In fact part of the reason I'm not unhappy about delaying this is that I can tell, with absolute confidence that, the minute add El Torito support support to Rufus, I'll start receiving complaints of "But it doesn't work for THIS specific ISO from manufacturer XYZ! ".

[sschuberth]

I'll start receiving complaints of "But it doesn't work for THIS specific ISO from manufacturer XYZ! ".

Does that mean you'd be opposed to accepting a contribution that adds this feature?

[pbatard]

Does that mean you'd be opposed to accepting a contribution that adds this feature?

Not at all. But I want it to be done "right" so I'll be pushing back contributions that only add support for a specific type of scenario (the one the submitter specifically wants to solve) and lack the genericity to address other scenarios.

If someone submits a pull request for this, then I will expect them to come up with a "This PR should work for more cases, including the ones I am not directly interested in, because I understand this is meant to be provided as a generic feature" rather than "I added extraction and patchup that I need for my specific case. You sort out the rest!".

In other words, I want people who may submit a PR for this feature to look a bit further than their immediate issue with the lack of El Torito support, and spend some time investigating other images and other extraction scenarios, because that's what I'm going to be doing when/if I add the feature.

[LBXComputers]

I have an ISO which has an El Torito structure. It appears to be a bootable floppy image written at the start of the ISO, much like say the Windows 98 CD has. There is no other content on the disc. Running the ISO via VirtualBox you can see the contents mounted in to A: which is where the autoexec is. It would be great to have this convert to USB so this comment is written to simply add my interest in having El Torito structure supported in a future version. I guess in the meantime, I need to somehow extract the boot part of the ISO only to get access to the files and then copy these on to a regular FreeDOS boot USB?

[pbatard]

I guess in the meantime, I need to somehow extract the boot part of the ISO only to get access to the files and then copy these on to a regular FreeDOS boot USB?

Yes, that's more or less what's described in this section of the FAQ.

[LBXComputers]

I guess in the meantime, I need to somehow extract the boot part of the ISO only to get access to the files and then copy these on to a regular FreeDOS boot USB?

Yes, that's more or less what's described in this section of the FAQ.

Ah thanks, I didn't spot that.

[tonymet]

FYI for future users [no help needed]

I followed the "seatools" instructions of FAQ. It took a few tries so here are the settings that worked
In Rufus ...
format with freedos
Advanced --> legacy partition

In your bios
secure boot = off
boot = legacy
1st boot device = usb

remember to restore your settings once you are done booting from usb

[marcosfrm]

Recent Ubuntu and Arch have moved the FAT image to a separate partition outside the ISO 9660 filesystem (keeping /EFI/BOOT there for the USB case -- thank God...):

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/DRFSPVZNNN4GVTAKU4RLIG2S57YWLKJ5/

https://gitlab.archlinux.org/archlinux/archiso/-/issues/49
https://gitlab.archlinux.org/archlinux/archiso/-/commit/729d16b48c99c5d9b23a89123ecde4ecacfa8705
https://gitlab.archlinux.org/archlinux/archiso/-/commit/09b6127fe8cabaf9a54a3bb864b0e3e009ca8476

[pbatard]

I'm going to consider that, with the number of people interested by this feature dwindling at about the same rate that BIOS/Legacy boot is still being used, the cost of adding this feature is no longer worth the benefits, especially as the detailed workaround I provide in the FAQ consisting of creating a DOS drive and extracting the boot image there, should work in most cases.

As such, I am going to close this feature request.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue if you think you have a related problem or query.