Is the Windows registry a good idea?

Compared to Unix config files, and even to Windows 3.1 ‘ini’ files is the windows registry a good idea? This was the question that presented itself in my mind this weekend.

First, what is the windows registry? Windows registry is a Hierarchical database that stores configuration settings for Windows, and for the software installed on your machine.

Now, coming back to the original question I feel while at the core, the registry is a good idea – keeping every setting in a centralised place – it encourages non-portable software. If you install, say, CorelDraw at a certain location on your harddisk, and want to move it to a different location, you need to uninstall it and then reinstall again. Ditto for moving from one computer to another. A time-consuming process – especially if you use multiple computers – one at work, one at home, and another at a cafe.

Why is it that I say the registry, per se, encourages non-portable software? The reason is, when you had config files, and you moved software, the files moved with it too. With registry this doesn’t happen. Note however, either way its possible to write portable or non-portable software. Just that, when a programmer writes code without portability as a key focus, and uses config files, the end product is more likely to be portable.

What can be done about it? When a software program runs, it needs to check if the keys it needs are present or not. If not, it should try to default the parameters, and add them to the registry.

One current approach to writing portable software, is to use config files and provide an update program: if you move the software, you run the update program. This program will detect the file-paths and update those in the config files. The same can be used for registry.

Share

NTFS: Alternate Data Streams

Not many people working on Windows using the NTFS file system are aware of a feature called Alternate Data Streams (ADS).

First things first: A file system is that part of an operating system which deals with storage and management of files on the disk. Its responsible for the ability to retrieve data from a file when needed. An operating system can support multiple file systems – for example, Windows XP supports at least FAT and NTFS.

Now we come to ADS. Its a feature of NTFS, and allows data to be stored in “hidden” blocks linked to a file. Normal operations such as reading or writing to the file do not reveal the existence of this data. So it can happen that your hard disk is full, yet the file sizes do not reveal the reason. Here is a small ‘test’ I did in this regards (for people who understand command line – a more ‘Windows’ example is available here):

C:\>copy con hello.txt
This is visible to all.
^Z
        1 file(s) copied.

C:\>dir hello.txt
 Volume in drive C has no label.
 Volume Serial Number is A8EB-38D4

 Directory of C:\

10/15/2008  12:56 PM                25 hello.txt
               1 File(s)             25 bytes
               0 Dir(s)   8,501,428,224 bytes free

C:\>echo This is visible only to some > hello.txt:hiddendata

C:\>type hello.txt
This is visible to all.

C:\>dir hello.txt
 Volume in drive C has no label.
 Volume Serial Number is A8EB-38D4

 Directory of C:\

10/15/2008  12:56 PM                25 hello.txt
               1 File(s)             25 bytes
               0 Dir(s)   8,501,428,224 bytes free

C:\>more < hello.txt:hiddendata
This is visible only to some, and is part of ADS

This is what I did: I created a file called hello.txt with some text. Thereafter, I checked the file-size and it was 25 bytes. I then added some text to the file, but under an ADS named ‘hiddendata’. I again checked the file-size: it was still shown as 25 bytes, and the contents did not show the content of the ADS. However, when I directly asked to be shown the content of the ADS, it was shown to me.

Windows may use ADS to store access related information, or anything it wants to. In particular this is used to store “zone” information – when a file is downloaded from the internet and you see a warning when executing the file – its because of this zone information stored in the ADS. To remove ADS from a file, a quick way is to copy it to a FAT device and then copy it back. Since FAT cannot hold ADS, it gets deleted.

Virus writers, however, may use this feature for anything: they can store their own files without anyone noticing. Can you run an executable file directly off an ADS? You bet:

C:\>start /b c:/hello.txt:hello.exe

C:\>

Hello, World.

Hence it’s important to keep an eye on your NTFS drives. One way to do this is to use a tool called LADS. It comes with a very nice FAQ on ADS. Another tool, one that I haven’t tried myself is LNS.

The official word from Microsoft is here, and another interesting PDF is here.

LADS can be run on a partition, say C: as below:

lads c: /s

If you notice anything strange beyond “Zone.Identifier” or “Thumbs.db:encryptable” – especially a file with ‘exe’ extension – it can be a cause for concern.

Share

Deleting zero byte files

In the past I have shown ways to run Linux scripts on windows based system. I have also talked about one use of the find command, in conjunction with grep command to search for files having a given text content (say a word) in multiple folders.

Today I will show you another use of the find command: to automate tasks such as deleting zero byte files. This is a pretty common cleanup task that’s carried out on machines that are involved in EDI file transfers.

Here is the script:

find . -size 0 | sed -e 's/^/rm /' | sh

It deletes all zero byte files in the current folder, and in the folders below it.

In order to understand this, you may need to read about pipes. The task is carried out in three steps:

  1. find . -size 0 searches for all zero byte files in the given folder and the folders below it and returns the filepaths.
  2. sed -e 's/^/rm /' turns the list of names into a script – for example if the name is ‘/data/x’ it changes it to ‘rm /data/x’. More on sed here.
  3. the last steps simply forwards the script to the Linux shell for execution.

This is also a very flexible script and can be customised to carry out a wide variety of tasks. Please post your variations as comments.

Share

Automate encryption with GPG

Privacy
Privacy

This blogpost requires some familiarity with GPG.

Today I want to share the scripts for running GPG in the batch (unattended) mode. You can have a password on the keys if you want, but since this is the automated mode, you may want to use keys without a password. Irrespective of whether or not you use a password – use a separate set of keys that you will not use for anything that not batch processed. The scripts are primarily for Linux. If you need them for Windows, please read this and post any problems you face in comments section, and I will help.

The script for encryption is here:

#! /bin/sh
GNUPGHOME='/apps/gpg/'
export GNUPGHOME
gpg --batch -r <reciepient> --output $2 --passphrase-fd 3 --sign --encrypt $1 3</apps/gpg/passph

The first line shows the location of the shell – please change it according to where the shell is on your system. The second line has the location of the folder where the keys are located – the pubring.gpg and secring.gpg. In the last line, replace <reciepient> with the name on the reciepient key. The /apps/gpg/passph points to the location of the file containing the passphrase. The script will both sign and encrypt the file – change this to suit your needs. The script expects the name of the input file and the name of the output files as parameters in that order.

The script for decryption is here:

#! /bin/sh
GNUPGHOME='/apps/gpg/'
export GNUPGHOME
gpg --batch --passphrase-fd 3 --status-fd 1 --decrypt $1 3</apps/gpg/passph | grep '\[GNUPG:\] GOODSIG'
if [ $? -eq 0 ]; then
  gpg --batch --passphrase-fd 3 --output $2 --decrypt $1 3</apps/gpg/passph
fi

The first three lines are similar. Line 4 just checks if the file has a valid signature. If you want to skip this step remove lines 4, 5 and 7. No effort is made to see who signed the file. In my scenario, I could control the people who could sign by having only those public keys in the repository, and the repository could only be written to by ‘root’.

Please post comments in case of questions, concerns.

Share

Searching files within multiple folders

How to search file contents for a specific phrase using grep within multiple folders. Shows ways for both Linux and Windows.

On Linux, the normal way to search for some text within a file is to use grep (Global Regular Expression Print). However, grep has a limitation: it cannot automatically search folders within the current folder. It can only search within files in the current folder. Today I will show you how to use grep to search within all files and folders inside a current folder (recursively).

Windows users – despair not. If you find the standard windows search brain-dead or want to automate the task through scripts, you can also use this script. I have already explained various ways to run Linux scripts on windows – use the one that suits you.

We will couple grep with the find command, to unleash the power.

Here is what you need to do on Linux (or Cygwin):

find . -type f -exec grep -iH 'dedicated' {} \;

and use this for UnxUtils on Windows:

find . -type f -exec grep -iH "dedicated" \"{}\" ;

This does a case insensitive search for the word ‘dedicated’ in the current folder and all subfolders under it. Change -iH to -H for case sensitive search.

You can read the manuals for find and grep and change the commands to suit your needs – this method provides a lot of flexibility. Post your precise usage in comments, especially for Windows.

Share

Running Linux scripts on Windows

How to run Linux scripts on Windows.

Running Linux shell scripts on Windows is very useful to me. We may need to tweak some scripts depending on the environment in use.

One of the best ways to run them, is within Cygwin. This is something I have already written about.

If Cygwin is too bulky for you, you can use UnxUtils. This is a small set of widely used Linux Utilities, and includes the zsh shell. If you use this you will need to limit the scripts to those that use the commands available within this set. However, it still packs in a lot of paunch.

You might need to ensure the file path syntax expected by the given script matches the environment. Cygwin supports both forward-slash and backward-slash. However, please check the documentation/test to see what works.

Last option, is to run Linux on Windows using Virtual PC. That’s the best bet if you can invest the time needed to do the setup. Instructions are available here. However, this will not be able to access your Windows file system. If that is important for you (like it is to me in most cases), please stick to Cygwin. On the other side, if you are testing code that will eventually run on a Linux machine, the VPC method is better.

Share