Automatic data communication as a development strategy.
To get a lead on your competitors.
To avoid duplication of effort.
To reduce costs.
To reduce boring and repetitive work.
To avoid having to remember technical details.
Automatic communication is both for professionals and amateurs. It helps
keep your costs down. Also, it lets you do the job faster and safer. Sometimes,
it is the only way to get the job done.
We all have different needs
Automation will never be the same for everybody. Our needs are too different.
 Some get excited when a program
can dial a bulletin board, retrieve a program, and then disconnect without
having to touch the keyboard.
Some want an "answering machine"
that can respond to and forward email when he or she is away from the office.
Others want a communications
system that can tap selected news sources, search databases, and do
post-processing on the retrieved material.
For most professionals, doing
things manually takes too much time. Time is better spent reading, digesting,
and using, rather than on stupid technical retrieval work. Computers can
do that.
To others again, automation
is a question of being able to use the online resource at all. If it takes
60 seconds to get a piece of information, it may be possible to get before
running for the next meeting.
However, if it takes 15 minutes,
there may not be enough time. If you must also read a help text to find out
how to do it, you may not even consider doing it. Your mind is so full of
other things right now.
When using a system for automatic communication, you do not have to learn
and remember online commands. The system will do it for you.
The minimum solution
Automatic data communication in its simplest form entails the following:
-
One keystroke to get the communications program to dial a number, and send
user name/password when the online service requests this information.
-
Macro commands (like in a word processor) for navigating through an online
service, searching, and to send complex commands by pressing one key.
Most communication programs have a macro language or a script language. There
are also macro programs designed to be used with Microsoft Windows. In many
instances, users are taking advantage of these features without even knowing
it. This is the case for many of those connecting to the Internet with a
simple mouse click.
If this has not been done
for you, then you will probably never regret time spent on learning how to
use these features. At a minimum, you should be able to have your system
log on automatically to your favorite service provider.
Autologon spares you the task
of remembering your user name and password. Besides, most people are only
able to use the keyboard at a low speed. They easily get frustrated by having
to correct typing errors.
Auto-logon to the Internet
Trumpet Winsock is a popular program used to set up a TCP/IP protocol connection
with the Internet. It can dial up an Internet access provider, logon, and
set up a SLIP, PPP, or a pseudo-SLIP connection to the net using SLiRP or
The Internet Adapter (TIA).
You can use Trumpet by entering
commands manually, but most users prefer to let the program do it for them.
This is achieved by having all commands in a file called LOGIN.CMD. Trumpet
reads this file when you click on Dialer, Login, and then logon to your service
as instructed.
We call LOGIN.CMD a script
file. It is a text file that can contain a list of commands for dialing and
navigating on an online service. You can write and edit it using Windows
Notepad, DOS EDIT, or another utility that allows you to save the file as
DOS or ASCII text. (Most popular script languages have the same requirement.)
Trumpet's scripts can contain
commands that test for the occurrence of small pieces of information sent
by the access provider at given times (like the prompt "Password?").
When this information is found,
Trumpet can be set to send the proper response or command (in this case,
your secret password).
Let us look at a typical Trumpet
LOGON.CMD file. Comments are written in lines above the given commands preceded
with a #.
# The output command tells Trumpet to send text to
# the modem, in this case "ATZ" to reset the modem.
# The "\13" refers to ascii character number 13
# (Return). The effect is the same as if you were
# typing ATZ followed by a press on the Return
# key.
output atz\13
# The input command tells Trumpet to wait for the
# preceding string, here for "OK" followed by "\n"
# (short hand for \10, the Line Feed character).
input 10 OK\n
# The next 'output' command sends a modem setup
# string. Note that this string is particular to
# my modem. You may well need other commands.
# Also, note that the "\" character has a special
# function in Trumpet. Therefore, AT commands like
# "AT\N5" must be written as "AT\\N5".
output at&C1&d2\\V2\\N5W1S9=12S11=50\\K3\\Q2 &S1\13
# Waiting for another "OK" from the modem.
input 10 OK\n
# Sending autodial command and phone number to the
# modem
output atdt,22568862\13
# Waiting for the "CONNECT" report from the modem
# when a connection with the remote modem has been
# set up.
input 50 CONNECT
# Wait until it's safe to continue sending. Some
# modem's hang up if you transmit characters
# during a connect set to monitor the carrier
# signal detector (DCD).
wait 30 dcd
# Now, prod the remote terminal server by sending
# a Return
output \13
# Wait for the username prompt ('login:').
input 30 login:
# Enter your username. In this example the user
# name is 'Presno'.
output Presno\13
# Wait for the password prompt
input 30 sword:
# Enter the secret password. In this example the
# password is "secret"
output secret\13
# we are now logged in
I use this script to dial a PPP server. When Trumpet signals "PPP ENABLED,"
I can open Netscape for the World Wide Web, WinVN for reading news, or Pegasus
for mail.
Trumpet uses a variant of
this script when calling in to a Unix host to use SLiRP (or The Internet
Adapter). In these applications, it waits for the host's prompt (@gaia%)
before starting the SLiRP application to get PPP (or SLIP). This is achieved
by adding the following commands after "# we are now logged in" above.
# The @ and % characters are rather unique, so
# instead of 'input 30 \64gaia\37', I just use
input 50 \37
# Finally, the pseudo-SLIP startup command
output slirp -P\13
input 20 SLiRP Ready ...
Trumpet signals "PPP ENABLED," and I am ready to go.
Auto-logon with Procomm
Procomm is a general communications programs. Many other programs, like Qmodem
and Telix, have equivalent capabilities.
When writing a script for
auto-logon to a new service, your first step is to list the commands that
you believe required. Enter them in a text file (as DOS or ASCII text).
Procomm, Qmodem and Telix
let you list your favorite online services in a dialing directory, and tie
these entries to your scripts. Press a key to start the appropriate script
file for automatic access to a service.
This simple PROCOMM script
used to work for to access my now defunct bulletin board in Norway. It assumes
that your name is Jens Mikkelsen, and that the secret password is FOXCROOK4.
You'll have to change this before testing.
;
;Script file for auto-logon to SHS
;
WAITFOR "our FIRST Name? "
PAUSE 1
TRANSMIT "Jens^M"
WAITFOR "our LAST Name? "
PAUSE 1
TRANSMIT "Mikkelsen^M"
WAITFOR "ots will echo)? "
PAUSE 1
TRANSMIT "foxcrook4^M"
WAITFOR "^JMore (Y),N,NS? "
PAUSE 1
TRANSMIT "n^M"
WAITFOR "^JMore (Y),N,NS? "
PAUSE 1
TRANSMIT "n^M"
WAITFOR "R] to Continue? "
PAUSE 1
TRANSMIT "^M"
The commands are different from Trumpet, but easy to understand. Here is
an explanation:
-
the ";" character at the beginning of a line identifies it as a comment line.
Procomm is to ignore it. We use such lines for notes.
-
WAITFOR "our FIRST Name? " has Procomm wait for the text string
"our FIRST NAME?." It is a part of the question "What is your first
name?"
-
PAUSE 1 halts the execution of the script file for one second.
-
TRANSMIT "Jens^M" sends the name "Jens" followed by a RETURN
(the code ^M in Procomm).
-
WAITFOR "our LAST Name? " makes Procomm wait for the question
"What is your LAST Name?"
The script continues like this. In WAITFOR commands, we use part of the text
that is displayed on our screen once the scrolling stops.
Make sure that the search
term is unique. It must not appear elsewhere in the text coming from the
host computer. If it does, your name and password may be sent too early.
You can call the script
HORROR.CMD, and attach it to the entry for my board in your Procomm phone
directory. When you call it the next time, Procomm will execute the commands
in the file and "turn the keyboard over to you" when done.
Macros
Above, we used a script to log on automatically to a service. When Procomm
gives us access to the keyboard again, we must continue manually.
What we want to do online
varies. Sometimes, we want to read new messages in conferences. In other
cases, the purpose is to check new programs in the file library. If we find
programs of interest, we may want to download them.
Shorthand macros can help
you do this faster and safer. For example, one macro can take you quickly
to a conference for new messages. You can make Procomm start this macro whenever
you press ALT-0 (keep the ALT key down, then press 0).
You can have the macro key
ALT-1 send other commands when in the file archives.
When I started using MS-DOS
computers for data communications, PC-TALK became my favorite program. It
has many of the same macro capabilities that Procomm has.
With PC-TALK, I did autologon
to the now defunct NewsNet service. Macro number one sent commands that gave
me the contents of various newsletters. Macro #2 picked up the contents in
another group. Macro #3 picked up stories from my mailbox, and macro #4 logged
me off the service. My mission was completed by pressing four or five keys.
The bookmark files of Netscape's
Navigator and Microsoft Explorer are also nice macro features.
Automating the full task
It's a long way from automated logon scripts and the use of macros to full
automation. The major difference is that with full automation, you do not
have to look at the screen while the script is working. You can do other
things. Sometimes, you may not even be present when the job is being done.
On a typical morning in my
house, I go directly from bed to my office to switch my communications computer
on.
While I visit the bathroom,
my communications program calls two Internet service providers to retrieve
and send information.
When the script has disconnected,
it analyzes the received data. I want important mail highlighted. Some
information is to be automatically appended to storage files. Retrieved news
menus are to be analyzed to see if there are any articles that I want to
read (compared with a set of keywords). "Spam" messages (unsolicited
advertisements) are to be deleted automatically, etc.
Sometimes, the unexpected
happen. There may be noise on the phone line, or a sudden disconnect. Usually,
my script can solve this without manual intervention. It is therefore allowed
to work unattended most of the time.
When I get to my office after
breakfast, it is all done. My communications program is set for reading and
responding to today's email. I can sit down, and immediately get to work.
After having written all my
replies, I say "send" to my system. For me, it's time for another cup of
coffee. I am not needed by the keyboard while mail is being sent.
This is what an automatic
communications system can do. My scripts also help plan and prepare online
visits, and ease my work by postprocessing results.
When your communication is fully automated, you need not read incoming
data while it scrolls over your screen, and then again after logging off
the service. You do it only once.
How to get it? Here are some alternatives:
Alternative 1: Write your own system
You can write procedures for powerful script-driven programs like ProYam
(from Omen Technology) and Crosstalk MK IV for DOS.
I started writing scripts
for ProYam way back in 1985. The system
is constantly expanded to include new services, refined to include more
functions, and enhanced to become more robust.
My system work like an autopilot.
It calls online services, navigates, retrieves, sends data, and logs off.
Postprocessing includes automatic
reformatting of retrieved data, transfers to various internal databases,
statistics, usage logs, and calculation of transfer costs.
Such scripts can do quite
complex operations online. For example, it can
-
Buy and sell stock when today's quotes are over/under given limits,
-
Select news stories and other types of information based on information found
in menus or titles.
Script writing is not for everybody. It is complicated, and takes a lot of
time, so it may not be for you.
On the other hand, those going
for it seldom regret, and you may find systems offered by others that you
automate more of your work. Tailor-made communication scripts give a wonderful
flexibility. The software does not cost much, but it sure takes a lot of
time!
Hints for script writers:
Do not use large and complex script files before you know the online service
well. The scripts let you do things quicker and safer, but there is always
a possibility for unexpected problems.
Test your scripts for
a long time to make them robust by "training" them to handle the unexpected.
Leave them to work unattended when you are reasonably certain that they can
do the job. - It may take months or years to get to that point.
Build a timeout feature
into your scripts, so they do not just hang there waiting for you after a
meeting with fate.
Alternative 2: Use scripts made by others
Some script authors generously let others use their creations. Earle Robinson
of CompuServe's European Forum, share his ProYam
scripts for automatic usage of CompuServe with others. They are available
from the PC Communications Forum library (GO PCCOM).
Enter GO XTALK on CompuServe
to find advanced script files for Crosstalk Mk.4.
ZCOMM and ProYam scripts for
visiting my board automatically can be freely downloaded there. They split
access up into these three phases:
| Phase 1 |
|
Menu driven offline preparation. |
| Phase 2 |
|
Automatic logon, navigation through the system, and automatic
disconnection. |
| Phase 3 |
|
Automatic offline postprocessing. |
You will find scripts for other programs on many online services.
Alternative 3: Special software
Several online services sell communication programs with built-in functions
that provides you with automation. They can have offline functions for reading
and responding to mail. The degree of automation varies.
There are also many programs
written by third parties. Most programs assume that you use 'expert' as your
default operating mode on the online service.
TapCIS, NavCIS, Autosig (ATO),
OzCIS, CISOP, CompuServe Navigator (for Macintosh), CSNav/Win, AutoPilot
(for Amiga), ARCTIC (for Acorn Archimedes), and QuickCIS (for Atari) are
popular choices on CompuServe. The DOS program
TapCIS is my personal favorite. (CIM does not offer much automation!)
Journalist is an interesting
program that creates a personalized newspaper view of CompuServe for Windows
users. It automatically logs on to retrieve the information necessary to
fill the frames in your document, and formats it according to your
specifications. (Email: 71333.2163@compuserve.com)
LEXIS-NEXIS News Plus has pull-down menus and
detailed selection of commands. This MS-DOS program helps users set up detailed
search commands before logging on to the LEXIS-NEXIS service. Your search
results will be downloaded automatically.
Alternative 4: Offline readers
The alternatives above have one important weakness. Noise on the line can
prevent the "robot" from doing the job. All it takes is for noise to give
a prompt another content than is expected by your program or script (as in
"En@er a number:" instead of "Enter a number:").
You can avoid noise problems
by using grab or get commands (Chapter 12), and by
making the online service use its minimum prompts ('expert mode') . Still,
this does not give full protection.
The best is to let the online
service do the navigation. Think of it as logging on to run a batch file
on the remote computer. Combine this with automatic transfers of your commands,
sent in of one stream of data with automatic error correction (in the software
and in the modem), and you have a very robust system.
The program logs on to the
service. Then the service takes over. It registers your user identity, checks
your user profile for personal interests, retrieves and packs all messages,
news and files into one compressed file, and sends it to you at high speed.
Your outgoing messages, search
commands, commands to join or leave conferences, and more, are transferred
to the remote computer in a similar packet (compressed file).
When received by the remote
computer, it unpacks the transfer file and distributes messages and commands
to various services following your instructions.
Your "physical" contact with
the service is when your modem is disconnected. The help menus that you read
belong to your program, and not the online service. You read and respond
to mail in a reading module (ref. the term "offline reader").
Some offline readers give
the caller access to more tools than is available on the online service itself.
They may have spelling checkers, multimedia support, let you use your favorite
editor or word processor, and offer various storage, search, and printing
options.
They may let you sort incoming
conference mail by "threads," and permit you to place obnoxious writers on
a "kill" list.
Using offline readers is probably
the easiest, cheapest, and safest way of using online services. You even
risk writing more concise and thoughtful messages. The "readers" are popular
among bulletin board users, and some commercial services are also starting
to accommodate them.
There are many offline reader
programs. The most advanced take over completely upon logon, and manage transfers
of commands and compressed information files to and from the host. (Example:
Binkley Term on FidoNet)
Global Link is an offline
reader for EcoNet. Bergen By Byte offers the BBS/CS Mail Grabber/Reader,
a script system used with the communications program Telix and the service's
"auto-get" function.
NUPOP (MS-DOS), Eudora for
Windows (Windows and Macintosh), and WinQVT/Net (Windows) are offline readers
for use when calling Unix hosts.
The most popular systems on
the PCBoard based Thunderball Cave BBS are Offline Express, Megareader, Session
Manager, Rose Reader and EZReader. Freddie is a reader for Macintosh.
These readers are used with
scripts written for various communication programs. Some of them have built
in communications (and script) modules.
EZReader from Thumper Technologies
(P.O. Box 471346, Tulsa, OK 74147-1346, U.S.A.) lets users retrieve mail
from several online systems using transfer formats such as QWK, PCBoard capture
files, ProDoor ZIPM files, XRS, MCI Mail, and others.
1stReader from Sparkware (Post
Office Box 386, Hendersonville, Tennessee 37077, U.S.A.) is my personal favorite
for accessing Qmail based online systems.
Note: Some offline readers contain all the features required for
fully automated communications. Some bulletin boards allow up- and downloading
to start right after CONNECT.
Off-Line Xpress, an offline
mail reader for QWK (Qwikmail) packets, does not contain a communications
module. It just does pre- and postprocessing of mail packets.
You can use the Off-Line
Xpress as one element in a larger automated system. For example, a system
for access to PCBoard bulletin boards may consist of Off-Line Xpress software,
PKZIP and PKUNZIP (popular shareware programs to compress/decompress mail
packets), the QMODEM communications program, and a script to navigate to/from
the QWK packet send and receive area on the BBS.
1stReader contains a powerful
script based communications module. It let you compose replies, set search
commands, subscriptions to services, add and drop conferences, and enter
download commands offline.
Automatic automation
We have explained how to write scripts with Procomm. However, there are simpler
and quicker ways. Many communications programs can make scripts automatically
using a learning function. It goes like this:
Start the learning function
before calling the online service. Then log on, navigate to the desired services,
do what you want to automate, and disconnect.
The learning feature analyzes
the received data and builds a script file for automatic communication.
If you call again with the
new script, it will "drive the same route one more time."
ZCOMM and ProYam have a learning
feature. This is how I made a script for accessing Semaforum BBS using ZCOMM:
ZCOMM asked for a phone number. I entered +47-370-11710. It asked for
speed, and I entered 14400 bps. Next, I had to choose one of the following:
(1) System uses IBM PC (ANSI) line drawing
(2) 7 bits even parity
(3) 8 bits no parity
My choice was 1.
ZCOMM dialed the number.
When the connection was set up, I entered my name and password, navigated
to the message section, read new messages, browsed new files in the library,
and entered G for Goodbye. This was the "tour" that I wanted to automate.
When disconnected, I pressed
the F1 key. This prompted the learning process based on a record of the online
tour. The log described everything that had happened in detail, including
my pauses to think. Now I was prompted by the following question:
'newscr.t' exists. Append/replace/quit?
I selected append. Then:
Do you want this script file as a new entry in your
telephone directory (y/n)?
I entered "y," and named it "semaforum." After a few seconds, my new script
was ready:
Your new script is in the file 'newscr.t' !! You can append the
file to your current script file (for example PHODIR.T) or have the commands
executed by entering:
call semaforum.newscr.t
It was time to test the new wonder. I entered
call semaforum.newscr.t
at the ZCOMM command line, hit the ENTER key, and off it went. ZCOMM called
the BBS and repeated everything - at far higher speed than I had done it
manually. It went on-hook as planned when done.
Limitations
Auto-learn programs can create a script file that let you "drive the same
route." For some applications this is enough. For others, it's just part
of the way. You have to refine the script manually to get what you want.
Example:
If you some bulletin boards with an auto-learned script made yesterday,
chances are that everything works well. If you call twice on the same day,
however, you may be in for a surprise as the board may greet you differently
on your second visit. If this is the case, then your script must take this
into account.
On most online services, many things can happen at each "junction of your
road." At one point in one of my scripts, up to twenty things may happen.
Each event needs its own "routing."
Twenty possible events are
an extreme, but three to four options at each system prompt is not unusual.
All of them need to be handled by your script, if you want it to visit online
services unattended while asleep.
It is quicker and simpler
to use other people's scripts and programs, although this might force you
to use a different program for each service.
Personally, I prefer offline
readers on services where they can do the job. On other services, I usually
depend on my own tailor-made scripts. |
