Identifying Hardware

CPU Architecture

Installing Gentoo Linux reminds me that there are times when a GNU/Linux user can benefit from knowing the details of their hardware. uname is a great command to accomplish this.

$ uname -a # a for all information
Linux livecd 2.6.19-gentoo-r5 #1 SMP Thu Apr 5 15:05:48 UTC 2007 i686 Intel(R) Celeron(R) Celeron(R) CPU 2.93GHz GenuineIntel GNU/Linux

Here, uname tells us that a LiveCD is running with Linux kernel version 2.6.19-gentoo-r5. It's compiled for i686 multiprocessor systems. The processor in use is a Intel Celeron and it's clocked at 2.93GHz.

PCI Bus

Another great command for learning about your hardware is lspci. If your Linux kernel isn't using some of the hardware on your computer, you may find that lspci won't list any details about those certain pieces of hardware, however. Here's an example of it in use:

# lspci -v
00:00.0 Host bridge: ATI Technologies Inc R200 AGP Bridge [Mobility Radeon 7000 IGP] (rev 05)
Flags: bus master, 66MHz, medium devsel, latency 64
Memory at b0000000 (32-bit, prefetchable) [size=64M]
Memory at b4000000 (32-bit, prefetchable) [size=4K]
Capabilities: [a0] AGP version 2.0

For brevity, I've only included the first piece of hardware found on the PCI bus. We can also list the hardware in a tree, which is great for seeing what pieces run through various busses.

# lspci -tv

The -v switch just makes the output a little more verbose.

Searching

Finding Files with an Index

Linux provides an easy way to create an index of files for fast future searching. updatedb would have been used in the past, and still can be, but a newer implementation just uses it as an alias to call slocate with a switch. What's slocate? It stands for secure locate and is like the original locate command, which searches through the index for a given file, except that it doesn't list files that one has no permission to view.

# slocate -u # update file index

Once the file index is up to date, one can find any given file quickly using the following command.

$ slocate filename # locate any file with filename in its name

You may wish to review the man page for slocate as well because this command supports things like case insensitive search and regular expressions!

Finding Files without an Index

But what if we don't have an index of which to search through for the desired files? We can still use the find command. Here's an example:

$ find / -iname filename

Here, find is instructed to start searching at the root of the file tree for files with the name of filename, while ignoring their case.

Binary Files

whereis is a great command for locating binary files that are stored in a standard path. For example, if we want to find where the ls command is:

$ whereis ls
ls: /bin/ls /usr/share/man/man1/ls.1.gz

This tells us that the ls command has the path of /bin and the man pages are located in /usr/share/man/man1/ls.1.gz.

Finding Commands

There are times when one may wish to find a suitable program for a given task. For example, I may wish to find commands that deal with searching.

$ apropos search
apropos (1) - search the manual page names and descriptions
badblocks (8) - search a device for bad blocks
bzegrep (1) - search possibly bzip2 compressed files for a regular expression

Here apropos shows us that apropos itself, badblocks, and bzegrep can be used to search for different types of things. Remember, if you install new programs with new man pages, be sure to run mandb to update the list that apropos searches through.

Linux I/O Redirection

Redirection with Files

Let's get started with a few simple examples of redirection:

command > filename # Redirect command's output to a new or overwritten file
command < filename # Use filename's contents as standard input for command

The first command would overwrite an existing file. So what if we want to only append command's output to a file?

command >> filename # Append command's output to filename

If you're a C++ programmer, you can remember the difference between this and the first output redirection command by recalling that C++ uses the >> operator to add to a stream, not replace it.

Speaking of C++, it uses three main streams:
stdin: 0
stdout: 1
stderr: 2

If we want to save standard error to a file, we can:

command 2> error_output

We could do the same with standard output, or even redirect standard error to standard output:

command 2>&1

And that's the basics of using I/O redirection with files. What if we want to send the output of one command to another as it's input? Linux allows us to do this using pipes.

Pipes

We can pipe the output of one command into the input of another:

first_command | second_command

We could even create a named pipe that delivers one command's output to another as input. The cool thing about named pipes is that they allow programs in different terminals and among different users to communicate. As Linux treats named pipes as ordinary files, we can even set their permissions!

mkfifo name_of_pipe # create a named pipe
command > name_of_pipe # direct output to named pipe as if it's a file
command < name_of_pipe # accept input from named pipe; can be done as different user

Now you have the basics of redirection with files and pipes! Have a nice day :)

C++ Reference
Who's sick of always having to think about C++ syntax instead of getting into the flow of coding an idea? I am, so I put a reference sheet together. If you'd like a copy please feel free to email me at my full name in lower case and without spaces at gmail (dot) com.

Hero of Allacrost: Engine Overview

Until today, I would have had to shamefully admit that I a have barely considered another person's code beside my own. I'm not talking about example code. Rather, I'm referring to looking at another person or group's large-scale program and attempting to figure out how they structured it at a high level. If you haven't done this yourself, I can say from my my recent experience that it's an eye opener for considering different ways of doing things. Most importantly it allows one to see how a successful program is designed.

Today I'll be reviewing Hero of Allacrost. It's an up and coming C++ RPG in the Open Source world. Although it's not a full game as of yet, a tech demo has been released and the team is well on it's way to expanding it into a full story-driven game.

Originally when I had decided to look into how an RPG game engine was designed, I thought it would require slogging through code multiple times in a chaotic attempt to understand the engine's structure. Thanks to the work by some easy to understand authors gaining insight into their engine turned out to be rather painless, much to my relief.

Allacrost makes extensive use of the singleton design pattern. The defining point of singleton classes is that they can only be instantiated once. Attempting to instantiate them again common returns a pointer to the already instantiated instance. Allacrost has audio, video, scripting, input, global, mode, and system singletons, among others. They are instantiated in main.cpp so that whatever uses them later on doesn't have to worry about setting them up. One can pretty much guess what these singleton's do, based on their names, except for the mode singleton, which leads us to the next defining point of the Allacrost engine.

Allacrost uses a concept of game modes. Example modes include map exploration, battling, shopping, paused gameplay, the menu screen, and even quit confirmation. Multiple modes can exist at the same time on a stack, with the mode at the top being the one that the game actually runs. Each mode requires three functions:

• Reset(): Prepares a mode, which has just reached the top of the mode stack, for use again.
• Update(): Allows a mode to do its processing.
• Draw(): Draws a mode.

The real power of modes is evident in the idea that the engine can be expanded to include new modes in the future.

Speaking of updating a mode, Allacrost uses time-based movement as opposed to fixed movement. This allows the game to take advantage of fast computers while at least running on older computers. From reading the Allacrost Quick Start Guide, I learned that the sleep function, which would be required with fixed movement on fast computers, has a varying accuracy across different operating systems. Besides any modern operating system will divide processing power between the running processes. Allacrost still makes use of the sleep function while in pause mode due to it's low use of the processor.

The input singleton accepts input from a keyboard or joystick and provides it to the rest of the engine in a queue of commands, such as move up. Each command has three accessors: pressed, released, and state. A command is pressed during the first main loop iteration that it is detected in. Released is similar. That leaves state, which is only true if the button triggering a given command is currently down.

The audio singleton makes use of the OpenAL and Ogg Vorbis libraries. It treats sound and music similarly, but distinctly and with differing features. While one song can play at a time, there may be a number of sound effects in use at once. To keep things simple, as all programs that are actually completed must, Allacrost refers to music and sound files by filename, excluding path and extension. All music files are stored in a single directory, while sound files are stored in another. There are no category folders for audio beyond that. Audio can be played, paused, stopped, and unloaded, among other things.

Next up, we have the video singleton. OpenGL is used for graphics, DevIL for images, and SDL_ttf for fonts. One common problem game programmers face is what the heck to do with different screen resolutions. Is a game's difficulty balanced really balanced when you can see more of the map just because you are using a higher screen resolution than someone else. Allacrost gets around this by dividing the screen with a coordinate system. The video singleton has a function that allows the programmer to choose the coordinate system, so for example, the screen can be 32 arbitrary units across and 24 arbitrary units high. The Allacrost team uses this to ease tile placement in map mode. Another point of interest with the video singleton is that it permits drawing on the screen using a cursor that can have it's location set absolutely or relatively, among other things. Relating to the video singleton, the GUI singleton uses unicode for displaying text so that the game can be translated to languages of other alphabets in the future.

What RPG would be complete without a scripting language? Allacrost wouldn't and that's why it uses the Lua scripting language. Of interesting note, Lua is also the Roman goddess whom soldiers would offer their seized weapons to. Anyway, the Lua scripting language can be bound with C++ allowing the Allacrost team to rapidly create game content to fill their engine with. Scripts can be stored in a binary or text mode. It makes no difference to the engine, although binary scripts will naturally run faster. The engine can read them using functions like bool readBool (const char *key) and int readInt (const char *key). The key just identifies what variable in a given table (as Lua calls a scope) should have it's value returned.

Finally, one more topic I feel is of merit in this summary of the Allacrost Getting Started Guide is error handling. The Allacrost team uses their own Exception class, upon which all other exceptions are derived from. If an exception unwinds the stack all the way back to the main game loop, the game crashes with a message. Smaller errors, such as failing to load a file, are handled immediately through the boolean return value of related functions.

With the above said, Hero of Allacrost is definitely a game to keep an eye on. Even if it goes nowhere, the existing documentation and code is something to learn from! Best of luck to the development team. You guys, and girls if there are any, rock!

Monitoring the Load in Linux

uptime

If you'd like quick information about the amount of work your system is doing, look no further than uptime. This handy little program displays the current time, system uptime, number of current users, and load averages over the last 1, 5, and 15 minutes.

Everything is pretty strait forward, except for load averages. A load average is the average number of processes that are using the CPU or waiting for something such as IO or even the processor itself, among other things.

Here's an example of uptime in action:
$ uptime
14:14:38 up 4:13, 2 users, load average: 0.30, 0.44, 0.44
Here, the time was 2:14pm. My system has been up (running) for 4 hours and 13 minutes. There are currently two users logged in, and for about a third of the past 1, 5, and 15 minutes, one process on average has been using my CPU or waiting for something else.

top

While uptime specifies the amount of work one's system is doing, top describes just what work is being done. Here's an example session of top, exited with q for quit:

top - 11:16:42 up 1:34, 3 users, load average: 0.63, 0.65, 0.40
Tasks: 108 total, 3 running, 105 sleeping, 0 stopped, 0 zombie
Cpu(s): 15.2%us, 1.7%sy, 0.0%ni, 82.9%id, 0.0%wa, 0.0%hi, 0.2%si, 0.0%st
Mem: 450816k total, 443132k used, 7684k free, 28236k buffers
Swap: 497972k total, 34704k used, 463268k free, 99940k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6176 samuel 16 0 266m 96m 27m R 10.2 21.9 5:22.29 firefox-bin
6310 samuel 15 0 135m 35m 19m S 3.6 8.1 1:22.64 rhythmbox
5187 root 15 0 212m 51m 7560 R 1.7 11.6 2:56.52 Xorg
6206 samuel 15 0 81316 20m 11m S 0.2 4.6 0:03.92 gnome-terminal
6426 samuel 15 0 2368 1156 884 R 0.2 0.3 0:00.04 top
1 root 18 0 2948 1852 532 S 0.0 0.4 0:01.21 init

Notice the first line is the output of uptime? The column headings a little way down, starting at the left, describe the Process ID of a given process, the USER who appears to own the process, it's priority, nice value (modifyable priority), amount of VIRTual memory in use, physical ram RESources in use, SHaRed ram in use, process State, percent of the CPU in use, percent of ram MEMory in use, minutes:seconds.hundredths of seconds that the process has used the processor for, and the name of the process itself. To learn more about what top can do for you be sure to press h while it's running to get a list of hotkeys. If you make any changes, you may wish to use the W hotkey to save them for next time.

dstat

dstat provides handware usage statistics for a variety of systems as one can see in the following example:

$ dstat

----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system--
usr sys idl wai hiq siq| read writ| recv send| in out | int csw
24 6 68 2 0 0 | 95k 27k| 0 0 | 0 10k| 422 1091

Regarding the CPU, we can see usage based on the user, system, and the amount of the processor that isn't being used at all. We can see disk, network, paging, and system access as well.

free

If you just want to know how much ram is free and swap space (the overflow of ram that is stored on disk), free is the command to use! The only extra switch I recommand is -m which displays the output in megabytes instead of the kilobytes which is the default.

total used free shared buffers cached
Mem: 440 432 8 0 24 167
-/+ buffers/cache: 240 199
Swap: 486 0 486

In case you're wondering the shared column is obsolete. Buffers just refers to the memory in use by the Linux kernel itself.

pstree

pstree is a great little command that displays a tree of the child processes created by each parent process. Here's an example:

init─┬─NetworkManager───3*[{NetworkManager}]
├─NetworkManagerD
├─acpid
...

Linux Firewalling

iptables is Linux's modern firewall, implemented within the kernel. It can monitor different types of packets, on different paths, and process them in various ways.

Chains are an important first concept in learning iptables. They contain lists of rules that are matched against packets and targets that describe what to do with the packet. A target can be a user-defined chain, ACCEPT, DROP, QUEUE, and RETURN (which directs the packet back to the chain that called this one). If the packet can't be matched to any rule in the chain, the policy decides the ultimate target.

A set of chains make up a table. Depending on one's kernel configuration and loaded modules, different chains may exist. Without anything special, one could expect the following tables to be present:

• filter, the default table, has the built-in chains: INPUT, FORWARD (for packets routed through the computer), and OUTPUT.


• nat, for packets that create new connections.


• mangle, for specialized packet alteration.


• raw, mainly for allowing certain packets to bypass connection tracking.

Argument Summary

Really, this entire article is a summary of man iptables, but this is especially true for the following commands, which represent only a portion of the vast array available in iptables:

• -A, --append chain rule-specification


• -D, --delete chain rulenum


• -I, --insert chain [rulenum] rule-specification


• -L, --list [chain]
: Displays basic information about the chains in a given table.

• -F, --flush [chain]: Deletes all the rules from a given chain.


• -Z, --zero: Zeroes out the various counters.


• -N, --new-chain chain


• -X, --delete-chain [chain]: Without a given chain, all custom chains in the table are deleted.


• -P, --policy chain target


• -s, --source [!] address: A name or IP address is acceptable.


• -d, --destination [!] address: A name or IP address is acceptable.


• -j, --jump target: A jump isn't required if counting packets is the only desired effect.


• -v, --verbose: Shows the packet and byte counters for each rule, among other things.


• --line-numbers: Shows the rule number of each rule in the chain that's displayed.

So how does one go about blocking a website, for example, using iptables? Like this:
iptables --table filter --append OUTPUT --destination www.digg.com --match owner --uid-owner samuel --jump REJECT --reject-with icmp-host-unreachable
Here we are working on the filter table's OUTPUT chain. Every outbound packet originating on this computer must travel through this chain of rules. The destination we are attempting to match in this rule is www.digg.com. Optionally, we only care if I'm the one initiating this connection. Instead of just dropping the packet, which would cause the browser to hang for a while, it is clearly rejected.

Persistence

Ah, now I can finally get some work done :) But wait, when you reboot your computer the firewall rule will be gone. What we need to do is save the firewall rule so that we can restore it upon system startup:
iptables-save > /root/firewall_rules # Initially save the firewall rules.
By the way, you may wish to try iptables-save on it's own first just to make sure that there are no firewall rules that another program has currently setup, which would otherwise mingle with your own. Now fire up your favorite text editor to create a startup and shutdown script. I'll use GEdit:
gedit /etc/init.d/firewall
Paste the following into the new file and save it:
case "$1" in
start|"")
echo Loading firewall rules...
iptables-restore < /root/firewall_rules
;;
stop)
echo Saving firewall rules...
iptables-save > /root/firewall_rules
;;
esac
And that's it! Now your custom firewall rules will persist between sessions. Happy Hacking!

Comparing Files in Linux

sdiff, much like it's counter part diff, provides a great way to compare files in GNU/Linux! A good mnumonic to remember the command is by thinking of it as the "side-by-side merge of file differences." Here's an example of how it works.

First, create two files, beginning with textfile1:
Apples
Oranges
Bananas

... and the second file, textfile2:
Apples
Oranges
Strawberries
Bananas

With two files to compare, let's run sdiff on them:
$ sdiff --width 10 textfile1 textfile2
Apples Apples
Oranges Oranges
> Strawberr
Bananas Bananas

sdiff contains a few additional features of interest:

• --output=FILE


• --ignore-case


• --ignore-all-space

And that's it, so remember: When you need to quickly compare files in Linux, sdiff is a convenient method!