Interfacing QM with Linux Email

Interfacing QM with Linux Email

5/23/06 - Doug Dumitru

Background:

I had a client that maintained a mid-size contact database on a QM server. The database included a field with email addresses. The database was being maintained with a QM/Coyote web-based application.

The client wanted an easy way to send mass emails to all of the email addresses in the database. As is typical, the client tools for sending emails consited of an ISP supplied mail application with no mass mailing capabilities.

The Solution:

I setup an inbound mailbox that would deliver mail to QM instead of to a Unix mailbox. QM would then strip extra headers off of the mail message and resend the message to every record in the database with a valid email address.

The Server:

This application is hosted on a public-facing Linux server running Fedora Core 5. It is actually a "Xen" virtual server, but as far as applications running inside of the server, it is a dedicated box.

Sendmail Configuration:

The only MTA (Mail Transport Agent) software on the server is sendmail, definately not the most plesant to configure. In this case, the setup was so simple, that sendmail was fine.

Step 1: Setup a domain name to point to the server.

Mail requires DNS to run. If you have a simple domain name, or a subdomain, you can just point the domain to the IP address of the server. If you want to setup an 'MX' record you can, but this is not required. We gave the customer a throwaway record under one of our "infrastructure" names:

customername.easyco.net      IN A 1.2.3.4

Step 2: Setup sendmail to recognize the customer's domain name as a local domain.

This involves adding 'customername.easyco.net' to /etc/mail/local-host-names.

Step 3: Setting up sendmail for inbound mail.

The default configuration for sendmail only listens on 127.0.0.1 (the loopback address). In order to accept mail from the outside world, you need to change this to 0.0.0.0. This is done in /etc/mail/sendmail.mc line:

DAEMON_OPTIONS(`Port=smtp,Addr=0.0.0.0,Name=MTA')dnl

Step 4: Recompile the sendmail configuration file.

With Fedora Core this is easy. Just cd to /etc/mail and run 'make'. After you rebuild the sendmail configuration file, you should tell sendmail to reload it's configuration with '/etc/rc.d/init.d/sendmail reload'.

Step 5: Open the firewall

In order for mail to move, you need to open port 25 both inbound and outbound.

Step 6: Setup an 'alias' for the recieved email address.

The address that will relay is a "special" address that does not have a user associated with it. In this case, it is also "private", so we picked an address that would not get "guessed" by the spammers. You then enter this address at the end of /etc/aliases

custname-db-873466542:      "|qm all"

This setup allows you to send email to custname-db-873466542@customername.easyco.net and the mail message itself will be delivered to the 'qm.sh' script.

After you add an alias, remember to run the 'newaliases' command to rebuild the aliases database.

Step 7: Setup the 'qm.sh' script

Sendmail delivers email thru scripts in /etc/smrsh. In this case, a 5-line bash script was used to invoke qm.

#!/bin/bash

MSG=/tmp/$$.txt
cat - > $MSG

cd /usr/qmaccount
/usr/qmsys/bin/qm -quiet EMAIL-FWD $1 $MSG >> /var/log/qm.sh
     
rm $MSG

QM Programming

So now we have sendmail delivering email messages to QM. It does this by running the QM program EMAIL-FWD passing two parameters on the command line. The first parameter is a command that will tell EMAIL-FWD what to do. Perhaps there are several lists attached to several email addresses, so this is a way to re-use the EMAIL-FWD program for several purposes. The second parameter is the Linux path to a temporary file that contains the email message itself.

The EMAIL-FWD program.

This is a pretty simple QM/Basic program:

   rem EMAIL-FWD

$catalog local

   cmd = field(sentence(),' ',2)
   MsgId = field(sentence(),' ',3)


   o = object('Email.cls')
   err = o->ReadMsg(MsgId)
   if err <> '' then abort err
   err = o->StripHdrs
   if err <> '' then abort err

   cmds = ''
   cmds<1> = 'all'
   cmds<2> = 'test'

   t = listindex(cmds,@am,downcase(cmd)) + 1
   on t goto cmd.undef,
             cmd.all,
             cmd.test

   abort

cmd.undef:

   abort 'EMAIL-FWD undefined command'

cmd.all:

   gosub GetAll(name.lst,email.lst)

   for i = 1 to dcount(name.lst,@am)
      err = o->SendMsg(name.lst<i>,email.lst<i>)
   next i

   gosub SendSummary('Admin person 1','addr1@example.com',name.lst,email.lst)
   gosub SendSummary('Admin person 2','addr2@example.com',name.lst,email.lst)

   stop

cmd.test:

   gosub GetAll(name.lst,email.lst)

   err = o->SendMsg('Admin person 1','addr1@example.com')
   err = o->SendMsg('Admin person 2','addr2@example.com')

   gosub SendSummary('Admin person 1','addr1@example.com',name.lst,email.lst)
   gosub SendSummary('Admin person 2','addr2@example.com',name.lst,email.lst)

   stop 

LOCAL SUBROUTINE GetAll(name.lst,email.lst)

   PRIVATE db.fd
   PRIVATE name,emails,id,d

   open 'DATABASE-FILE' to db.fd else abort 'Cannot open DATABASE-FILE'

   name.lst = ''
   email.lst = ''

   execute 'SSELECT DATABASE-FILE BY SNAME' capturing dummy
   loop
      readnext id else exit
      read d from db.fd , id else continue

      name = trim(d<1>:' ':d<2>)
      if name = '' then continue

      emails = trim(d<15>)
      if emails = '' then continue

      emails = change(emails,',',@am)
      for i = 1 to dcount(emails,@am)
         email = emails<i>
         if count(email,'@') <> 1 then continue
         name.lst<-1> = name
         email.lst<-1> = email
      next i
   repeat

   return

LOCAL SUBROUTINE SendSummary(to.name,to.email,name.lst,email.lst)

   PRIVATE s.msg,i,err

   s.msg = 'Subject: Message Summary'
   s.msg<-1> = 'From: "DATABASE-FILE  database server" <' : o->FromAddr      : '>'
   s.msg<-1> = 'To: "Me" <addr1@example.com>'
   s.msg<-1> = ''
   s.msg<-1> = 'The folllowing addresses were used:'
   s.msg<-1> = ''
   for i = 1 to dcount(name.lst,@am)
      s.msg<-1> = ' ' : name.lst<i> 'l#30' : ' ' : email.lst<i>
   next i

   o->SetMsg(s.msg)

   err = o->SendMsg(to.name,to.email)

   return

end

This program uses a class named 'Email.cls':

   CLASS Email.cls

$catalog global

PRIVATE Msg
PRIVATE FromAddr
PRIVATE cr,tab

PUBLIC SUBROUTINE create.object

   cr  = char(13)
   tab = char(9)

end

GET ReadMsg(path)

   err = ''

   openseq path to src.fd else
      err<-1>= 'Email.cls: ReadMsg: Cannot open message source file ' : path
      return(err)
   end

   Msg = ''

   loop
      readseq l from src.fd else exit
      l = change(l,cr,'')
      Msg<-1> = l
   repeat

   return(err)

end

SET SetMsg(n.msg)

   Msg = n.msg

end

GET WriteMsg(path)

   err = ''

   openseq path OVERWRITE to dst.fd else
      create dst.fd else
         err<-1> = 'Email.cls: WriteMsg: Cannot open/create outbound message file ' : path
         return(err)
      end
   end

   i1 = dcount(Msg,@am)
   for i = 1 to i1
      writeseq Msg<i> to dst.fd else
         err<-1> = 'Email.cls: WriteMsg: Error writing outbound message file'
         return(err)
      end
   next i

   return(err)

end

GET StripHdrs

   err = ''

   if Msg = '' then
      err<-1> = 'Email.cls: StripHdrs: No message in queue.'
      return(err)
   end

   l.msg = Msg
   Msg = ''
   FromAddr = ''

   CurState = ''

   i1 = dcount(l.msg,@am)
   i = 1
   loop
      if i > i1 then exit
      l = l.msg<i>
      begin case
         case CurState = ''
            if l = '' then
               Msg<-1>= ''
               CurState = 'body'
               i += 1
               continue
            end
            if downcase(field(trim(l),' ',1)) = 'from' then
               if FromAddr <> '' then
                  err<-1>= 'Email.cls: StripHdrs: Duplicate From line.'
                  return(err)
               end
               FromAddr = field(trim(l),' ',2)
               i += 1
               continue
            end
            w = downcase(trim(field(l,':',1)))
            begin case
               case w = 'received'
                  CurState = 'skiphdr'
               case 1
                  Msg<-1> = l
            end case
            i += 1
            continue
         case CurState = 'hdr'
            Msg<-1> = l
            i += 1
            continue
         case CurState = 'skiphdr'
            begin case
               case l[1,1] = ' ' or l[1,1] = tab
                  i += 1
               case 1
                  CurState = ''
            end case
            continue
         case CurState = 'body'
            Msg<-1> = l
            i += 1
            continue
      end case
   repeat

   return(err)

end

GET ReplToHdr(val)

   err = ''

   if Msg = '' then
      err<-1> = 'Email.cls: ReplToHdr: Message is not loaded.'
      return(err)
   end

   l.msg = Msg
   Msg = ''

   CurState = ''

   i1 = dcount(l.msg,@am)
   i = 1
   loop
      if i > i1 then exit
      l = l.msg<-1>
      begin case
         case CurState = ''
            if l = '' then
               Msg<-1> = ''
               CurState = 'body'
               i += 1
               continue
            end
            w = downcase(trim(field(l,':',1)))
            begin case
               case w = 'to'
                  CurState = 'skiphdr'
                  Msg<-1>= 'To: ' : val
               case 1
                  Msg<-1> = l
            end case
            i += 1
            continue
         case CurState = 'hdr'
            Msg<-1> = l
            i += 1
            continue
         case CurState = 'skiphdr'
            begin case
               case l[1,1] = ' ' or l[1,1] = tab
                  i += 1
               case 1
                  CurState = ''
            end case
            continue
         case CurState = 'body'
            Msg<-1> = l
            i += 1
            continue
      end case
   repeat
   
   return(err)

end

GET SendMsg(ToName,ToAddr)

   err = ''

   if Msg = '' then
      err<-1> = 'Email.cls: SendMsg: No message is loaded.'
      return(err)
   end

   if ToName <> '' then
      s = '"' : ToName : '" <' : ToAddr : '>'
   end else
      s = ToAddr
   end

   err<-1> = me->ReplToHdr(s)
   if err <> '' then return(err)

   t.file = '/tmp/' : @userno : '_qm.txt'
   err<-1> = me->WriteMsg(t.file)
   if err <> '' then return(err)

   cmd = 'cat ' : t.file : '| /usr/sbin/sendmail -f' : me->FromAddr : ' ' : ToAddr
   * display 'cmd = ' : cmd
   os.execute cmd
   os.execute 'rm ' : t.file

   return(err)
end
   
GET FromAddr

   return(FromAddr)

end

GET Msg

   return(Msg)

end

end

Summary:

So there you have it. The setup needed to get sendmail to deliver mail to QM. Some QM to mangle the message headers, and then QM code to spit the message back out to hundreds of recipients. It is not optimized to be fast, but then again, I am not a spammer sending out millions of email messages. It should load on just about any current Linux distribution with sendmail and either commercial or GPL QM 2.4-4 or later. Mostly, it should help you get an idea of how sendmail and QM can cooperate sending and receiving email messages.

Because windows works very differently, moving this code to windows would be difficult.