AGI Specifications: Chapter 3 - AGI Internals
3. AGI internals
3.1 How the interpreter works
From the AGDS documentation, translated by Vassili Bykov (Last update: 27 January 1998).
The AGI interpreter contains:
- 256 8 bit variables, numbered 0--255 (Var);
- 256 flags, numbered 0--255 (Flag);
- a number of objects controlled by the interpreter, one of which (the object 0) may be controlled by the player using the keyboard;
- a number of inventory items;
- 12 40-character string variables (string).
Some variables (0 - 26) and flags (0 - 15) are reserved by the interpreter, all others are free to be used by the programmer.
The interpreter provides a common variable and flag space for all programs simultaneously loaded in the memory. The number of objects and things is determined by the OBJECT resource.
Interpreter's actions are described using the commands of the interpreter's language. For example, there are commands to manage objects, load and unload resources, etc. Further we shall consider the commands in detail.
Note: Any variable, flag, object, string, word, message, etc. has a unique ID number, and numbering of different data types is independent (for example, there can be a variable number 5, a string number 5, and a flag number 5).
Resources - the fundamental AGI data type
When we develop a game, we invent the plot, create objects of the game, animate them, develop scenery and a dictionary of words for the dialogue with the player. To describe all of these, resources are used. To create some of the resources, we use utilities included with AGDS, in this case the input of the utilities are resources. (Sounds weird, but that's literally what it says. --VB)
Here is a list of all the existing resources. Resources are used to represent:
- color background drawings (PICTURE resource);
- color animated images (VIEW resource);
- sound effects (music, noise) (SOUND resource);
- inventory items and other objects (OBJECT resource);
- system dictionary for communicating with the user (WORD resource);
- programs in internal AGI programming language (LOGIC resource).
General principles of the interpreter operation
Let us now consider the interpreter algorithm and the purpose of reserved variables and flags.
When interpreter starts, LOGIC resource number 0 is loaded in memory. It stays there during the whole play time and determines all the interpreter's actions related to the overall control of the game. The interpreter works in a loop, i.e. all its actions are described by the <a href="agispecs-03.html#workcycle">interpreter work cycle</a> shown in the block diagram below.
In each cycle the interpreter performs the following basic actions:
- time delay;
- clears the keyboard buffer;
- polls the keyboard and the joystick;
- analyses some of the reserved variables and flags (see block diagram);
- for all controllable objects for which animate_obj, start_update and draw commands were issued, directions of motion are recalculated;
- LOGIC resource 0 is executed, as well as any logics it calls--which, in turn, can call other logics. Depending on the state of variables and flags analyzed at step 4 the number of commands interpreted at stage 4 commands varies from one iteration of the cycle to another depending, for example, on a number of LOGIC resources to be called in the current situation;
- tests if the
new_roomcommand has been issued;
then the cycle is repeated.
All logics (programs and subroutines) simultaneously loaded in memory operate on a common set of variables, flags, and strings, each identified by a unique for each data type ID number.
The fact that the interpreter runs in a loop influences the general programming principles and style when programming for AGDS. This makes programming a little unusual and takes a certain time to get used to. For example, many cyclic activities requiring explicit loops in conventional programming languages are executed in the interpreter programs by default, provided the program has a proper structure.
General hints on how to reduce the time to adapt to the interpreter's language are given below, using an educational program <a href="agispecs-11.html#thunderstorm">Thunderstorm</a> as an example. However, this does not reduce the usefulness of analyzing the game programs of Sierra On-Line, Inc.
Interpreter work cycle
+---------------------------+ | 1. delay time | +---------------------------+ | V +----------------------------+ |2. clear the keyboard buffer| +----------------------------+ | V +---------------------------+ | Flag (2) - > 0 | | Flag (4) - > 0 | +---------------------------+ | V +-------------------------------------+ | 3. poll keyboard and joystick | +-------------------------------------+ | V +-------------------------------------+ | If the current mode is | | - program_control, the direction of | | Ego motion <-- var(6). | | - player.control, var (6) --> dir. | | of Ego motion. | +-------------------------------------+ | V +---------------------------------------------------------------+ | For all objects for which command animate.obj, start_update | | and draw were carried out, the recalculation of the direction | | of movement is performed. | +---------------------------------------------------------------+ | V +---------------------------------------------------------------+ | If the score has changed (var (3)) or the sound has been | | turned on or off (Flag (9)), the status line is updated. | +---------------------------------------------------------------+ | +---------+ V +--------------------------+ +---------------->| 4 Logic 0 is executed | | +--------------------------+ | | | V | +--------------------------------------+ | | - Dir. of motion of Ego <-- var (6) | | | - If score (var (3)) or Flag (9) | | | have changed their values - update | | | the status and score (on Var (3)); | | | - Var (5) - > 0; | | | - Var (4) - > 0; | | | - Flag (5) - > 0!!!! | | | - Flag (6) - > 0; | | | - Flag (12) - > 0. | | +--------------------------------------+ +----------------------------+ | | Execute: | V | ~~~~~~~~~~~~~~~~~~~~~~~~ | +------------------------+ | - stop.update; | | Update all controlled | | - unanimate.all; | | objects on the screen. | | - destroy all logic | +------------------------+ | resources except | | | for logic 0; | V | - player.control; | +--------------+ | - unblock; | | new.room n | | - set_horizon 36; | | or | | - var (1) = var (0); | | new.room.v n | | - var (0) = n | Var(n); | | was issued? | | - var (4) = 0; | | | | - var (5) = 0; | +-------+------+ | - var (9) = 0; |<-----------+ Yes | No | | - var (16) = number of | +-------+--+---+ | view assoc. w/Ego; | | | - Ego coords from var (2); | | | - var (2) = 0; | | | - flag (2) - > 0; | V | - flag (5) - > 1!!!! | +--------------+ | - score < - var (3); | | Go to step 1 | +----------------------------+ +--------------+
3.2 Variable types
Written by Lance Ewing, with additions/modifications by Claudio Matsuoka (Last updated: 22 May 1999).
There is a number of data types used as AGI command parameters, listed below:
Variable
This is an unsigned eight-bit variable type equivalent of a byte, or unsigned char. Its values range from 0 to 255. There are 256 variables and in the LOGIC code (listed in section <a href="agispecs-03.html#varlist">Variables used by the interpreter</a>); they are numbered from 0 to 255 and are identified by their number. (The original LOGIC source code that Sierra's programmers wrote would have had textual identifiers for these variables, but when the LOGIC source was compiled into the LOGIC codes, the original variable names were lost. To the interpreter, the variables are known by their index into the variable table.)
Variables are the most commonly used type. They feature in arithmetic commands such as addition and multiplication, and a lot of AGI commands have a version that has variable parameters as an alternative to the normal constant parameter versions.
Flag
Flags are the boolean type of the AGI system. Their value can be either 1 or 0 (true or false). There are 256 flags that are numbered 0 to 255. (In the original LOGIC source code, they would have had textual identifiers, but in the compiled LOGIC code they are known only by their index into the interpreters flag table.)
Flags are used to indicate when certain things have taken place.
String
According to another source, there are only 12 strings available. I don't know if this is true, but it agrees with the minimum amount of space set aside for strings that I have seen in examining memory usage during a game. However, the majority of AGI games have enough room for exactly 24 strings: AGI interpreter versions 2.089, 2.411, 3.002.107 and 3.002.149 have room for 12 strings, the remaining versions have room for 24 strings.
Whether the versions that have enough space for 24 strings do in fact support 24 strings is not known. Strings are 40 characters long which includes the zero terminator. String number zero is usually the input prompt (e.g. ">" or "]").
Word
Words are the words that the user types in. An input sentence is composed of a number of words. The important words (e.g. for the sentence look at the tree, look and tree are important) are assigned to the words variables corresponding to their place in the sentence once unimportant words and punctuation has been taken out. For example, in the earlier example word(1) would be look and word(2) would be tree. Words can be converted to strings.
Inventory Item
There are a number of AGI commands that refer to inventory items (e.g. get(), drop()). One of the arguments to these commands will represent an inventory item number. In the original LOGIC source text, the programmer would have written things like get(dagger) but the interpreter knows them only as an index into the OBJECT table.
Object
There can be a bit of confusion between this type and the inventory item because of the name of the object file. The object file has almost nothing to do with what the interpreter generally calls objects. There are a large number of AGI commands that deal with objects. For example,
In fact the interpreter calls its usage of the VIEW resource objects. An object is one usage of a VIEW resource. It is essentially an entry in the object table (or VIEW table/VIEW list). Many objects can use the same VIEW resource for its appearance which can be seen in KQ1 and BC with the crocodile filled moats.
So when an AGI command has an object as a parameter to it, the value of the parameter is an index number into a table of objects that the interpreter is currently controlling.
Message
At the end of every LOGIC file is a message section. There need not be any messages in it, but it will still exist. Messages in logic.0 are global messages whereas all other messages can only be accessed from their own LOGIC code. AGI commands that have messages as parameters refer to a message number in their own LOGIC file. I say that those in logic.0 are global because messages and strings can contain format codes one of which is used to display messages from logic.0.
Example:
print ("Message 34 in logic.0 is %g34.");
Therefore messages in logic.0 can be displayed by any LOGIC in this way.
3.3 Variables used by the interpreter
On interpreter startup all variables are set to 0.
- 0 - Current room number (parameter new_room cmd), initially 0.
- 1 - Previous room number.
- 2 - Code of the border touched by Ego:
* 0 - Touched nothing;
- 1 - Top edge of the screen or the horizon;
- 2 - Right edge of the screen;
- 3 - Bottom edge of the screen;
- 4 - Left edge of the screen.
- 3 - Current score.
- 4 - Number of object, other than Ego, that touched the border.
- 5 - The code of border touched by the object in Var (4).
- 6 - Direction of Ego's motion.
1
8 | 2
\ | /
\ | /
7 ------------- 3 0 - the object
/ | \ is motionless
/ | \
6 | 4
5
- 7 - Maximum score.
- 8 - Number of free 256-byte pages of the interpreter's memory.
- 9 - If == 0, it is the number of the word in the user message that was not found in the dictionary. (I would assume they mean "if != 0", but that's what they say. --VB)
- 10 - Time delay between interpreter cycles in 1/20 second intervals.
- 11 - Seconds (interpreter's internal clock)
- 12 - Minutes (interpreter's internal clock)
- 13 - Hours (interpreter's internal clock)
- 14 - Days (interpreter's internal clock)
- 15 - Joystick sensitivity (if Flag (8) = 1).
- 16 - ID number of the view-resource associated with Ego.
- 17 - Interpreter error code (if == 0) (Again I would expect this to say if != 0. --VB)
- 18 - Additional information that goes with the error code.
- 19 - Key pressed on the keyboard.
- 20 - Computer type. For IBM-PC it is always 0.
- 21 - If Flag (15) == 0 (command reset 15 was issued) and Var (21) is not equal to 0, the window is automatically closed after 1/2 * Var (21) seconds.
- 22 - Sound generator type:
- 1 - PC;
- 3 - Tandy.
- 23 - 0:F - sound volume (for Tandy).
- 24 - This variable stores the maximum number that can be entered in the input line. By default, this variable is set to 41 (29h). (information by Dark Minister)
- 25 - ID number of the item selected using status command or 0xFF if ESC was pressed.
- 26 - monitor type
- 0 - CGA;
- 2 - Hercules;
- 3 - EGA.
3.4 Flags used by the interpreter
On the interpreter startup all flags are set to 0.
- 0 - Ego base line is completely on pixels with priority = 3 (water surface).
- 1 - Ego is invisible of the screen (completely obscured by another object).
- 2 - the player has issued a command line.
- 3 - Ego base line has touched a pixel with priority 2 (signal).
- 4 -
saidcommand has accepted the user input. - 5 - The new room is executed for the first time.
- 6 -
restart_gamecommand has been executed. - 7 - if this flag is 1, writing to the script buffer is blocked.
- 8 - if 1, Var(15) determines the joystick sensitivity.
- 9 - sound on/off.
- 10 - 1 turns on the built-in debugger.
- 11 - Logic 0 is executed for the first time.
- 12 - 'restore_game' command has been executed.
- 13 - 1 allows the 'status' command to select items.
- 14 - 1 allows the menu to work.
- 15 - Determines the output mode of 'print' and 'print_at' commands:
- 1 - message window is left on the screen
- 0 - message window is closed when
ENTERorESCkey are pressed. If Var(21) is not 0, the window is closed automatically after 1/2 * Var(21) seconds
3.5 Memory organization
Written by Lance Ewing (Last updated: 31 August 1997).
The following information gives a rough guide as to how Sierra's AGI interpreter uses its memory. You can view this in operation in MS-DOS by using a memory resident program like Game Wizard.
Length of first data area (2 bytes)
Game signature (8 bytes)
Variables (256 bytes)
Flags (256 bits, 32 bytes)
Timers, blocks, and other special AGI variables
Strings (12*40 bytes or 24*40 bytes)
unknown
"Press ENTER to quit" etc
Script command jump table
"Avis Durgan" encryption string
Rest of AGIDATA.OVL is in here
unknown
words.tok file
object file
VIEW object table
logic.0
Other loaded resources
3.6 Game IDs and loaders
Written by Lance Ewing with additions/modifications by Peter Kelly and Anders M. Olsson (Last updated: 3 March 1998).
Since the data formats for the different AGI interpreter versions are mostly identical or easily convertible to each other, we should expect to be able to run one games data with another's interpreter. This sounds like a reasonable assumption but when you try it, the interpreter rejects the new data. The reason behind this is game IDs.
Every interpreter has got a game ID coded into it and it expects to be given data for that game. Somewhere in the initialization code for each game is a set.game.id(). When this command is encountered, the interpreter checks the given game ID and compares it with its own. If they are not the same, it quits immediately. Presumably the reason for this was to stop people running games with the wrong interpreter version, which can cause problems (unless you know what you're doing).
How do we get around it?
Method 1: The hard way.
Well, basically we have to find the game ID signature in the AGI interpreter file and change it to the ID of the game whose data we wish to be executed.
Here are a few examples of some game ID's and the data following them:
- Police Quest:
'P' 'Q' 0x00 'e' 'I' 'D' 'X' - Mother Goose:
'M' 'G' 0x00 'e' 'I' 'D' 'X' - Manhunter 2:
'M' 'H' '2' 0x00 'I' 'D' 'X' - XMAS Demo:
'X' 'M' 'A' 'S' 0x00 'D' 'X' - Leisure Suit Larry:
'L' 'L' 'L' 'L' 'L' 0x00 'X'
The game ID itself is the null terminated string that ends at the 00h. The text that follows it is of no significance, it is simply to fill in the gap although it is useful when searching for the game ID because, as you can see, this text is always "eIDX"> (presumably from gameIDX before being overwritten by the actual ID) or a suffix of it (i.e. "IDX", "DX", or "X"). For most games, the game ID is two or three characters which means that you will be able to rely on the "IDX" string being there for these games.
Method 2: The easy way
Using the above method will allow you to run a different game with an interpreter, but you will still only be able to run the game that has the specified game ID. There is another way around this, which involves patching the logic source. This can be accomplished using a program to edit the logics such as AGI Studio (for the Windows platform). What you can usually do is look at the top part of logic 0 and find out what the initialization logic is (usually around 90--100 -- you might have to look at a few logics before you find it). Then simply go to that logic, find the set.game.id command, remove it, and recompile the logic. Since the command is not used, the interpreter will not try and compare it with it's own ID, and it won't quit.
The only drawback to using this method is that the saved games no longer have the game ID in their name (so, for example, a savegame would be called sg.1 instead of kq2sg.1), but this is not a major hassle.
Why can't I find the game ID?
This is because a lot of the AGI files themselves are encrypted. See section <a href="agispecs-03.html#crypt">Encrypted AGI data</a> for further information.
Possibilities
What this means is that with a few useful AGI utility programs, it is possible to run any set of game data with a compatible AGI interpreter. For example, games that use AGI versions 2.915, 2.917, and 2.936 should be able to be converted into AGIv3 format and run with an AGIv3 interpreter.
Compatible as it is used above refers not only to the data differences but also to some AGI command dependencies. There are about four AGI commands that have changed the number of arguments passed to them as the interpreter developed. This sort of thing is the only real obstacle to running data on another interpreter.
3.7 Encrypted AGI data
Written by Lance Ewing, with additions/modifications by Peter Kelly and Anders M. Olsson (Last updated: 3 March 1998).
Many AGI files are encrypted. This was probably to give some protection to their product which was quite unique at the time. You can tell the difference between an encrypted AGI file and a non-encrypted AGI file by the first two characters. If they are MZ (the MS-DOS executable file header), then it not encrypted.
The AGI file is decrypted by the loader program. This is usually called sierra.com in MS-DOS but can also be named after the game (eg. kq1.com). In AGI version 1, the loader was called load. If an AGI game doesn't have a loader, then it shouldn't be encrypted. If an AGI game does have a loader, it does not necessarily mean that the AGI file is encrypted.
The decryption key was not originally embedded in the loader file. If you find a game where the key is embedded in the loader, it is because that game has had copy protection removed. There are several utilities to do that. Anders M Olsson's SUP is one of them. The CD re-releases have been unprotected by Sierra in exactly the same fashion.
The loader would read the decryption key from track 6 of the disk, load the executable file, decrypt and run it. Track 6 had a special format that was supposedly impossible to exactly reproduce by a standard PC floppy disk controller.
An interesting note is that when a copy-protected Sierra game asked for the original disk one, you could insert disk one from any protected Sierra game. The contents of track 6 were always the same.
But even though track 6 was the same, all games didn't use exactly the same encryption key. The two bytes in the loader, immediately following the string "keyOfs", gave an offset on track 6 from where the key would be loaded.
So, if the decryption string consists of 128 'k' characters, it can actually mean one of two things: Either the AGI file is not encrypted, or the game is still copy-protected.
How does the encryption work?
The loader contains a 128 byte string called the decryption key. Here's the process of decryption:
- The carry bit is zeroed.
- The first 128 bytes of the AGI file are XORed with the decryption key.
- The whole key string is rotated one bit to the right, including the carry-bit. (Bit 7 of the first byte is loaded from the carry.
- Bit 0 of the last byte is placed in the carry, where it will remain until the next rotation.
- The bit that was bit 0 of the last byte (now in carry) is ORed into bit 7 of the first byte.
- The next 128 bytes of the AGI file are XORed with the new key. If the end of the AGI file has not been reached, then go back to 3.
Where in the loader is the decryption key?
The start of a loader will look something like this:
There are two bytes in between the "keyOfs" string and the start of the description string. If the decryption key consists of 128 'k' characters, then the AGI file is not encrypted (or the game is still copy-protected). If it consists of a whole lot of random looking characters, then it is encrypted.
As an aside, the decryption string is followed immediately by the stack and is usually marked with a whole string of 's' characters. Thus we have 'k' for key and 's' for stack. The stack is usually 256 bytes long.
Decrypting the AGI file
Decrypting the AGI file is simply a matter of writing a program to read the loader to get the decryption string, and then applying the process mentioned above to the AGI file. Once this has been done, the game ID can be located.
3.8 Player input parsing
From the AGDS documentation, translated by Vassili Bykov (Last update: 31 August 1998).
Note: This section is an excerpt from the description of the said command from section <a href="agispecs-04.html#cmdref">Reference of LOGIC commands</a>.
Here is how the input is matched. After the player types a message and presses ENTER, the input line is processed by the interpreter in the following way:
- The interpreter removes all punctuation marks.
- All characters are converted to lowercase.
- All sequences of more than one space are replaced with a single space.
- Starting with the first word of the input, the interpreter looks up the vocabulary, trying to find the longest character sequence matching the input line.
If the search is unsuccessful, v9 is assigned the number of the word in the message that failed to match and the processing ends. If all the words have been assigned some codes:
- The Interpreter removes from the sequence of codes all zeros (that means all vocabulary words with zero codes are ignored).
f2(the user has entered an input line) is set to 1f4(saidcommand accepted the user input) is set to 0.
If the sequence of code produced by the interpreter is
V(1), V(2),...V(m)
The test is performed as follows:
If
f2== 0 orf4== 1, return FALSE.
Compare parameters W(i) and codes V(i) as follows:
- if W(i) = 1, it matches any V(i);
- if W(i) = 9999, it matches the whole remaining input i.e. the codes V(i), V(i+1),...V(m).
Otherwise W(i) should be equal to V(i).
If all elements match, f4 (said accepted the user input) is set to 1 and the command returns TRUE. Otherwise, FALSE is returned.
3.9 AGI interpreter versions
Written by Jeremy Hayes, with modifications by Claudio Matsuoka (Last update: 22 May 1999).
3.10 Version differences
Written by Lance Ewing (Last updated: 27 January 1998).
There are a number of different versions of the AGI interpreter but generally the data formats are the same or can easy be converted between each other. The following table is a list of AGI interpreter versions that I know of:
AGI
VersionInterp.
sizeAGIDATA
sizeNum of
CommandsObject file
encryptedLZW 2.089 34305 6656 155 No No 2.272 34816 6656 161 No No 2.411 38400 7680 169 Yes No 2.435 38400 7680 169 Yes No 2.439 38400 7680 169 Yes No 2.440 38400 7680 169 Yes No 2.915 39424 8192 173 Yes No 2.917 39424 8192 173 Yes No 2.936 39424 8192 175 Yes No 3.002.086 40866 8064 177 Yes Yes 3.002.098 40898 8080 181 Yes Yes 3.002.102 40898 8080 181 Yes Yes 3.002.107 40962 8080 181 Yes Yes 3.002.149 40520 7488 181 Yes Yes
This table illustrates a number of things:
- Firstly, as the interpreter version increased, the number of AGI commands supported increased with it. The last eleven we do not know the names of.
- There are two main AGI versions: AGI v2 and AGI v3.
- The early AGI v2 games did not encrypt the
objectfile with the Avis Durgan string. - AGI v3 games use adaptive LZW to compress their LOGIC, VIEW, and SOUND files.
Command argument number discrepancies
There are four commands that have changed the number of arguments that are passed to them. All this information is based on observations made of the above interpreter versions.
- The
quitcommand had no arguments for version 2.089 whereas all the others above have one argument. - The
print.atandprint.at.vcommands had only three arguments for versions 2.089--2.400 and four for the other versions. - Unknown command number 176 had one argument for version 3.002.086 but later versions had no arguments for this command.
Number of strings
There may be some differences in the number of strings supported by some interpreters as well. All interpreters have at least 12 strings. Most interpreters have space for 24 strings but I don't know if the extra space is used for strings or not.
< Previous: Chapter 2 - OverviewNext: Chapter 4 - The LOGIC Language >
