Big RamDrives and Beyond

From EDM2
Jump to: navigation, search

Original work by Micho Durdevich

Introduction

In this note we present a couple of new interesting solutions for remote booting diskless Win9x workstations from OS/2 Warp Server. These solutions could be considered as "complementary" to RIPLing methods discussed in our previous articles. The main idea here is to construct configurations with a sufficiently large RAM-drive [C:], capable to hold a micro-Windows "nucleus" package. This nucleus is actually a minimal Windows distribution with 32-bit networking. The rest of the Windows distribution is stored on the server, and becomes available to the client, after making appropriate network redirections in 32-bit mode. The same is true for various extra applications, which are all stored on the OS/2 server.

During the DOS boot phase, our RIPL workstations create a RAM-drive [C:] of an appropriate size, download a ZIPed Windows nucleus (something like an 8-16MB file) from the server, unpack it and prepare the RAM-disk for the use of long filenames.

The solutions we present here are faster, more reliable and run smoother than the previous RIPL configurations. The system is working in 32-bit mode, except the RAM-drive which stays 16-bit all the time. By construction, all the drives use long filenames.

The second part of the article discusses a very interesting problem of creating a superfloppy boot image, which is large enough to hold the whole Windows nucleus. This further simplifies our configurations in several ways. In particular, it is not necessary to make any special extra connection to the OS/2 server, in order to download the nucleus, and installation of such Windows requesters is simpler...

We also include a listing of a Win95 boot sector, and explain the structure of the BIOS Parameter Block of the boot sector.

Remarks

I presented one of these RIPL configurations at WarpTech 2000. The client workstation was an IBM NetVista (Network Station 2800) which RIPLed Windows 95 with full 32-bit networking, long filenames support and running Mathematica 3 in less than 2 minutes (all from an OS/2 Warp Server, with approx 45 seconds for the standard DOS RIPL-processing).

Preparing a Deployable Windows Nucleus

At first, carefully study what Windows does with its various files. Prepare a minimal Windows package with the properties

  • It is able to start Windows Shell consistently;
  • Full 32-bit Networking Support.

This minimal distribution should be something like 16MB of size (if you come down to 24MB that is acceptable). In case of Windows 98, we can also take advantage of the 98Lite philosophy.

Our next article presents an extreme version of a superfloppy-based clients, where the whole Windows starts from a 16-MB superfloppy. It also contains a detailed file-by-file specification of a Windows Nucleus.

Setting The Virtual Boot Floppy

The boot floppy that should be used here is a little different from the standard boot floppy used by DOS remote boot requesters.

First, there is no any networking program called from the AUTOEXEC.BAT. In standard configurations, the initial connection to the server is done using a CONNECT.EXE program and a stripped-down version of the IBM 16-bit networking (that fits on the floppy). After the initial connections

Z=\\servername\IBMLAN$ Y=\\servername\WRKFILES

are made, the system has a full access to the 16-bit networking package. In contrast to this, we put all the networking package on a virtual floppy, in a ZIPed form (it would not fit on a standard 1.44MB boot floppy). Alternatively, it is possible to use a larger boot floppy!

Overview of The Boot Process

Phase 1-Boot Block Processing

This is a standard boot phase for all remote boot requesters. The information about boot blocks is stored in CNF-files.

Phase 2-Starting Windows in DOS Mode

The system continues to boot from the corresponding virtual boot floppy, after RPLBOOT.SYS passes the control to RPLLOADR.COM. The appropriate RAM-drive [C:] is created, using a beautiful program XMSDSK by Franck Uberto. A Windows folder is created in [C:] that will contain the minimum set of files to run the IBM 16-bit redirection software.

The complete 16-bit networking package consists of the following files:

CMDS.EXE CMDS16.EXE NET.EXE NET.MSG NETH.MSG NETWORK.INI

We should change the line ripl=yes to ripl=no in the [network] section of NETWORK.INI. This will allow us to shut down completely the 16-bit networking, before loading Windows GUI. Also, the variable lanroot from the [network] section of NETWORK.INI should point to the IBM-networking directory in the RAM-drive.

Configurations discussed here have been also tested with the standard Microsoft RAMDrive, with basically the same results. The Microsoft RAMDrive is limited to 32MB, however.

Another excellent RAM disk driver is a shareware ReSizeable RAMDisk utility (SRDISK), by Marko Kothala. It can simulate many low-level hard disk and floppy disk properties, including the boot sector support, number of heads, cluster size, tracks format and number of FAT copies. It works very well in our context, and it is particularly useful in creating superfloppy configurations.

Phase 3-Downloading and Setting Up The Windows Nucleus

This is achieved by executing the batch file WIN-PREP.BAT. The file is listed in Appendix A. Basically, it does the following things:

  • Connects to the server and downloads a packed Windows nucleus;
  • Unpacks the nucleus (in the RAM-drive [C:]). Deletes the packed-nucleus file;
  • Downloads the client specific files, including the Registry and the appropriate INI-files;
  • Sets PATH, COMSPEC and attributes of critical files;
  • Adjusts the long filename attributes of the files within the Windows tree, with the help of DOSLFNBK utility (and using the backup file BACKUP.LFN of long filenames that is a part of the client-specific module);
  • Stops the RIPL service on the client.
  • Disconnects from the server and shuts down the 16-bit networking.

Phase 4-Starting Windows GUI

In this phase (which should be very fast) we boot Windows off the RAM-drive [C:], and realize extra network drive mappings (during Windows Logon) using Microsoft's 32-bit redirection software. For example, we can use one drive to hold the "full Windows" package (including for example the help files, Windows games and so on) and another drive to hold the apps.

4. Additional Remarks

M$ 16-bit Networking

Similar results can be obtained by using the M$ 16-bit networking in the DOS mode (instead of the IBM networking) and appropriately modifying the DOS-mode scripts. A slight complication is that we have to create the appropriate mini-registry file, and there are some subtle considerations involving PROTOCOL.INI and SNAPSHOT.EXE/VXD transition drivers. A very interesting topic, deserving a special attention, is the problematics of the dynamical transition from 16-bit to 32-bit networking.

A Couple of Details More

The RAM-drive [C:] consumes a considerable amount of the system memory (like 40MB). It is recommended to have a workstation of at least 64 MB of RAM. Normally, the swap file stays on [C:] (which means there is no effective virtual memory). Therefore, we should carefully plan what apps to run on the workstation, taking into account the effective RAM amount and the free space on [C:].

It is also possible to put the swap file on a network drive, but such configurations generate more network traffics...

Whenever we change something in the system configuration, the changes are not saved, unless we copy the Registry, the client-specific INI-files, and BACKUP.LFN (that holds the long filename info, it should be regenerated each time we actually change the long filenames structure) back to the workstation directory.

Some network adapters fail to reset to their base state, after shutting down the DOS-networking and RIPL service. This can cause some problems with Windows 32-bit networking initialization (locking the system). I found a nice trick: execute a basically empty command SNAPSHOT /M:2 before WIN!

A "hardware variation" of this is to play with two network adapters such that one of them performs the RIPL, and another one is used for 16-bit/32-bit networking (the second one is therefore a standard adapter, without the bootprom).

Instead of the above mentioned DOSLFNBK utility (by Duncan Murdoch, version 1.6 is freeware, later versions are shareware) we can use a somewhat diferent (and perhaps easier to use), but equally effective freeware utility DOSLFN by Chris Jones. Both programs could be used to prepare long filenames in [C:] during the DOS boot phase...

Constructing Superfloppy Images

In this section we shall first explain the structure of standard boot image files for DOS RIPL requesters of the OS/2 Warp Server. This will help us to see how to build non-standard virtual boot floppy images, that are up to approx 16MB in size. Such "floppies" are large enough to hold the whole nucleus package, which opens a possibility of further simplifying our Win95/98-Lite requesters. In particular this eliminates the need of using any redirection software in 16-bit mode (so the only 16-bit network software used consists of drivers loaded in the boot block file, during the initial RIPL processing).

It should be noted that the standard MAKEIMG utility does not allow us to create an image of something that is not interpretable as a real floppy disk!

The Structure of Standard Floppy Images

The structure of the standard floppy images is very simple: The image is a sector-by-sector copy of the corresponding bootable floppy disk, preceeded by a four-byte string. The sectors are ordered in the most natural way, starting from the boot sector and counting them on cylindar/head basis. A standard 1.44MB floppy possesses 80 cylinders, two heads and 18 sectors per track.

The above mentioned four-byte string that starts the image contains a basic information about the disk image. The string is read by RPLLOADR.COM at the end of the boot block processing. It has the following hexadecimal form

00 XY 01 ZT

where XYZT are hexadecimal digits. The second byte XY is the total number of cylinders minus one. The fourth byte ZT is the number of sectors per track. The maximal value for ZT is 3Fh which is (decimal) 63.

The Boot Sector Structure

The listing of a Win95 boot floppy boot sector is given in Appendix B. The structure of a hard disk boot sector is essentially the same. The first three bytes contain a jump+NOP instruction. The next part (003h-03Dh) is the so-called BIOS Parameter Block, containing the information about the disk low-level structure. Next, it comes the boot sector executable code (responsible for finding and loading IO.SYS in memory). After the executable code we find the data area (185h-1FDh). The boot sector ends with its signature - the word 55aa.

Here we shall explain in detail the BIOS Parameter Block. Multi-byte strings that represent numbers (not data) are always interpreted in the inverse-byte way-bytes are read from right to left.

  • OEM Field-8 bytes: A system description.
  • Sector Size-2 bytes: The size is in bytes. Normally the sector size is 512 bytes.
  • Cluster Size-One byte: The # of sectors per cluster. Must be a power of 2.
  • Reserved Sectors-2 bytes: Here, there is only one reserved sector (boot sector).
  • FAT Copies-One byte: Normally, there are two copies of the File Allocation Table.
  • Root Directory Entries-2 bytes: Normally, there are 512 root directory entries allowed.
  • Small Number of Sectors-2 bytes: This is enough for standard floppy disks...
  • Media Type-One byte: This is the description of the disk type (f0 in our case).
  • FAT Size-2 bytes: The number of sectors per FAT.
  • Track Size-2 bytes: Sectors per track. The maximal value is 3f.
  • Number of Heads-2 bytes: Normally, a 1.44MB floppy has 2 heads (sides).
  • Hidden Sectors-4 bytes: It is zero, except when for example there is a Partition Table with a Master Boot Record (a hard disk case).
  • Large Number of Sectors-4 bytes: Hard disks use this...(if used, the Small Number of Sectors should be zero, and vice versa).
  • Drive Identificator-One byte: It should be 00h for floppy disks, and 80h/81h for hard disks.
  • Reserved Byte: Should be left 00h.
  • BIOS Parameter Block Signature: Normally it is 29h.
  • Volume Serial Number-4 bytes: Displayed by VOL...
  • Volume Label-11 bytes: This is a data field.
  • File System Type-8 bytes: The description of the file system, normally it is FAT12/FAT16+3spaces; This is a data field.

Creating Bigger Images

Taking all this into account, it becomes possible to create "floppy disk" images of arbitrary size (within the framework allowed by the format). There are various ways to accomplish this task, depending on the software you use. Sometimes, it will be necessary to alter the BIOS Parameter Block...

In my experiments, I used SRDISK to create the appropriate RAM disk (working with a normal DOS-mode Windows workstation, with a HD). The SRDISK allows us to fine-tune all low-level disk parameters. I used the following DOS command (assuming the appropriate RAMDisk driver has been loaded in CONFIG.SYS):

SRDISK J: 15360 /A:2 /HEADS:2 /C:2048 /SECTORS:60 /BOOTSECTOR:<path>WIN95B.SEC

This creates a 15360 KBytes RAMDisk, with 2 FAT copies, 2 heads, 2048 bytes clusters, 60 sectors per track, and a predefined boot sector file. It is easy to see that such a disk should have 256 "cylinders" (the maximal number allowed by the OS/2 Warp Server boot image format).

The next step was to copy all necessary files, including the ZIPed Windows nucleus, user-specific files, the UNZIP program, the kernel IO.SYS together with COMMAND.COM and MSDOS.SYS, to such a RAMDisk. Also, I created the appropriate CONFIG.SYS and AUTOEXEC.BAT files. Note that the disk does not contain any 16-bit networking software.

Furthermore, I used another nice program, HDCP (Hard Disk Copy, by Chang Ping Lee) to create a linear sector-by-sector image of such a RAMDisk.

Finally, I transfered the disk image to the IBMLAN\DCDB\IMAGES directory of my Aurora Server (where boot images of DOS RIPL requesters are stored). Using Kon (a powerful binary editor by Bjorn Andersson) I added the corresponding 4-byte string

00 ff 01 3c

at the beginning of the image file. I changed the properties of a DOS requester representing my NetVista thin client, to point to a new "floppy disk" image...

It should come as no surprise that OS/2 Warp Server would work happily with all these funny virtual "floppy" disks, smoothly booting DOS RIPL requesters from them. However, it is not obvious that all DOS kernels would accept to deal with such non-standard formats. In my experiments, the kernel IO.SYS of the very first version of Win95 had problems to locate clusters on such floppy disks. It looks like this kernel preassumes some forms of a floppy disk and gets confused... However, later versions worked fine. It is still possible to use the very first version of Windows 95, but it is necessary to replace IO.SYS, COMMAND.COM and WIN.COM by later versions (and also, certain command line utilities).

It is possible to completely skip playing with an auxiliar DOS-computer, in creating a superfloppy image. We can work entirely within OS/2, using with several nice programs: Super Virtual Disk (SVDisk) by Albert Shane can be used to prepare and fine-tune the Windows boot superfloppy. Such a disk is then mapped to an image file. For example, by using DiskImage from Graham Utilities package, or PMDCopy by Jason Shannon. Finally, using a binary editor as Kon we can edit the image, changing the BIOS Parameter Block and completing the boot sector if necessary (one method of making the disk bootable, by the way!) and adding the appropriate 4-byte string.

Concluding Observations

Comparisions...

The described configurations have the following advantages over the solutions presented in the previous articles:

  • They are conceptually simpler and very elegant.
  • RIPL workstations boot faster and work faster.
  • Full long filename support.
  • From the very beginning, we skip many network access-related problems (like a very interesting icon problem, we solved in the first article by changing appropriate Registry settings, and re-creating the icon cache).
  • At least as stable as standard Windows systems.
  • Increased security.

There is also a purely academic reason to play with such RIPL configurations: In such a way we study the deeper architecture of Windows, and understand its truly modular structure. We also study the low-level disk structure. Simplicity is often inseparably related with Beauty and Coherence. As a paradigmic example from theoretical physics, we can mention general relativity theory, which is in its essence just a pure geometry. Cleaned up and simplified Windows looks much better, and runs much better. All this is along the lines of the 98Lite project.

On the other hand:

Our main boot drive [C:] will stay in the 16-bit mode all the time, simply because it is based on a 16-bit DOS driver XMSDSK/SRDISK. However, having in mind that this is a RAM-drive, it is actually much faster than any real hard drive, working in any mode.

Using DRVSPACE instead of ZIP/UnZIP

Another interesting variation of the construction is to prepare the main Windows nucleus as a DRVSPACE compressed drive.

In other words, the nucleus package has the form DRVSPACE.000 and it is transfered to a RAM-Drive during the DOS phase. For example, we can create two RAM-Drives, [C:] and [D:], and use the second one to hold DRVSPACE.000 only (in this case, loading the Microsoft RAMDrive.SYS twicely, or use RAMDrive jointly with XMSDSK/SRDISK). The compressed nucleus drive is then mounted simply by

scandisk D: /mount

This solution does not require any extra care about long filenames. It is also simpler to update the nucleus and it preserves more RAM, due to the inherent compression involved.

On the other hand, such RIPL workstations work a little slower, due to an extra compressing/decompressing load. The compressed volume file should be bigger than the ZIP-ed nucleus file, so it takes a little longer to transfer it from the OS/2 server.

Appendix A: A Sample WIN-PREP.BAT file

This sample file is suitable for configurations that connect to the server using the IBM 16-bit redirection. The script can be easily simplified for the superfloppy configurations...

@echo off 
echo Connecting to the OS/2 Server...
net use f: \\aurora\win95 /Y
echo Downloading Windows Nucleus...
copy f:\win4.zip c:\  >nul
copy f:\ziptools\unzip.exe c:\  >nul
set TZ=MARS
echo Unpacking... 
unzip -qq win4 
del win4.zip 
del unzip.exe
echo Initialization Image...
copy f:\c_init\*.dat c:\windows  >nul 
copy f:\c_init\*.com c:\windows  >nul 
copy f:\c_init\system.ini c:\win95  >nul
copy f:\c_init\win.ini c:\windows  >nul 
copy f:\c_init\control.ini c:\windows  >nul
copy f:\c_init\*.pwl c:\win95 >nul 
copy f:\c_init\*.bat c:\windows >nul
echo Setting Paths and the Command Processor... 
set comspec=c:\WINDOWS\COMMAND.COM
set path=c:\WINDOWS;c:\WIN95;c:\WIN95\COMMAND
echo Setting Attributes...
attrib +h +s c:\win95\fonts\DOSAPP.FON
attrib +h +s c:\win95\fonts\MARLETT.TTF
attrib +h +s c:\win95\fonts\VGAFIX.FON
attrib +h +s c:\win95\fonts\VGAOEM.FON
attrib +h +s c:\win95\fonts\VGASYS.FON
attrib +h c:\win95\SHELLNEW
attrib +h c:\win95\SPOOL
attrib +h +r +s c:\windows\SYSTEM.DAT
attrib +h c:\win95\TTFCACHE
attrib +h +r +s c:\windows\USER.DAT
attrib +h c:\win95\fonts\8514FIX.FON
attrib +h c:\win95\fonts\8514OEM.FON
attrib +h c:\win95\fonts\8514SYS.FON
attrib +h c:\win95\fonts\COURE.FON
attrib +h c:\win95\fonts\COURF.FON
attrib +h c:\win95\fonts\MODERN.FON
attrib +h c:\win95\fonts\SERIFE.FON
attrib +h c:\win95\fonts\SERIFF.FON
attrib +h c:\win95\fonts\SMALLE.FON
attrib +h c:\win95\fonts\SMALLF.FON
attrib +h c:\win95\fonts\SSERIFE.FON
attrib +h c:\win95\fonts\SSERIFF.FON
attrib +h c:\win95\fonts\SYMBOLE.FON
attrib +h c:\win95\fonts\SYMBOLF.FON
attrib +s c:\win95\fonts 

echo Setting Long Filename Attributes... 
cd\win95 
doslfnbk . /r /force
cd\windows
echo Terminating RIPL... 
rplterm 
echo Disconnecting From the Server... 
net stop requester /Y
echo Done!!! Ready to load Windows!

Appendix B: Win95 Boot Sector Listing

Here, we have used the symbol * do separate different parts of the boot sector (JMP+NOP, BPB, Executable Part, Data Part and Signature). The symbol | is used to separate different fields within the BIOS Parameter Block.

000:  eb 3e 90*4d 53 57 49 4e 34 2e 30|00 02|01|01 00|   ë>MSWIN4.0        JMP+NOP,
010:  02|e0 00|40 0b|f0|09 00|12 00|02 00|00 00 00 00|    à @ ð             Begin BPB
020:  00 00 00 00|00|00|29|14 00 b0 e2|4e 4f 20 4e 41          )  °âNO NA
030:  4d 45 20 20 20 20|46 41 54 20 20 20 20 20*f1 7d    ME    FAT     ñ}   End BPB,
040:  fa 33 c9 8e d1 bc fc 7b 16 07 bd 78 00 c5 76 00    ú3ɎѼü{  ½x Åv    Begin Program
050:  1e 56 16 55 bf 22 05 89 7e 00 89 4e 02 b1 0b fc     V U¿" ‰~ ‰N ± ü
060:  f3 a4 06 1f bd 00 7c c6 45 fe 0f 8b 46 18 88 45    ó¤  ½ |ÆEþ ‹F ˆE
070:  f9 fb 38 66 24 7c 04 cd 13 72 3c 8a 46 10 98 f7    ùû8f$| Í r<ŠF ˜÷
080:  66 16 03 46 1c 13 56 1e 03 46 0e 13 d1 50 52 89    f  F  V  F  ÑPR‰
090:  46 fc 89 56 fe b8 20 00 8b 76 11 f7 e6 8b 5e 0b    Fü‰Vþ¸  ‹v ÷æ‹^ 
0A0:  03 c3 48 f7 f3 01 46 fc 11 4e fe 5a 58 bb 00 07     ÃH÷ó Fü NþZX»  
0B0:  8b fb b1 01 e8 94 00 72 47 38 2d 74 19 b1 0b 56    ‹û± è” rG8-t ± V
0C0:  8b 76 3e f3 a6 5e 74 4a 4e 74 0b 03 f9 83 c7 15    ‹v>ó¦^tJNt  ùƒÇ 
0D0:  3b fb 72 e5 eb d7 2b c9 b8 d8 7d 87 46 3e 3c d8    ;ûråë×+ɸØ}‡F><Ø
0E0:  75 99 be 80 7d ac 98 03 f0 ac 84 c0 74 17 3c ff    u™¾€}¬˜ ð¬„Àt <ÿ
0F0:  74 09 b4 0e bb 07 00 cd 10 eb ee be 83 7d eb e5    t ´ »  Í ë}ëå
100:  be 81 7d eb e0 33 c0 cd 16 5e 1f 8f 04 8f 44 02    ¾}ëà3ÀÍ ^  D 
110:  cd 19 be 82 7d 8b 7d 0f 83 ff 02 72 c8 8b c7 48    Í ¾‚}‹} ƒÿ rÈ‹ÇH
120:  48 8a 4e 0d f7 e1 03 46 fc 13 56 fe bb 00 07 53    HŠN ÷á Fü Vþ»  S
130:  b1 04 e8 16 00 5b 72 c8 81 3f 4d 5a 75 a7 81 bf    ± è  [rȁ?MZu§¿
140:  00 02 42 4a 75 9f ea 00 02 70 00 50 52 51 91 92      BJuŸê  p PRQ‘’
150:  33 d2 f7 76 18 91 f7 76 18 42 87 ca f7 76 1a 8a    3Ò÷v ‘÷v B‡Ê÷v Š
160:  f2 8a 56 24 8a e8 d0 cc d0 cc 0a cc b8 01 02 cd    òŠV$ŠèÐÌÐÌ Ì¸  Í
170:  13 59 5a 58 72 09 40 75 01 42 03 5e 0b e2 cc c3     YZXr @u B ^ âÌÃ
180:  03 18 01 27 0d*0a 49 6e 76 61 6c 69 64 20 73 79       '  Invalid sy   End Program,
190:  73 74 65 6d 20 64 69 73 6b ff 0d 0a 44 69 73 6b    stem diskÿ  Disk   Begin Data
1A0:  20 49 2f 4f 20 65 72 72 6f 72 ff 0d 0a 52 65 70     I/O errorÿ  Rep
1B0:  6c 61 63 65 20 74 68 65 20 64 69 73 6b 2c 20 61    lace the disk, a
1C0:  6e 64 20 74 68 65 6e 20 70 72 65 73 73 20 61 6e    nd then press an
1D0:  79 20 6b 65 79 0d 0a 00 49 4f 20 20 20 20 20 20    y key   IO      
1E0:  53 59 53 4d 53 44 4f 53 20 20 20 53 59 53 80 01    SYSMSDOS   SYS€ 
1F0:  00 57 49 4e 42 4f 4f 54 20 53 59 53 00 00*55 aa     WINBOOT SYS  Uª   End Data,
                                                                            Signature