lessons learned while writing a java mailserver l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Lessons Learned while Writing a Java Mailserver PowerPoint Presentation
Download Presentation
Lessons Learned while Writing a Java Mailserver

Loading in 2 Seconds...

play fullscreen
1 / 36

Lessons Learned while Writing a Java Mailserver - PowerPoint PPT Presentation


  • 372 Views
  • Uploaded on

Lessons Learned while Writing a Java Mailserver by Richard O. Hammer about me 70s – BSEE, coded in Fortran and APL. 80s – went to CS graduate school at UNC-CH, ABD. Started a building business. 90s – started a libertarian think tank.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Lessons Learned while Writing a Java Mailserver' - bernad


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
about me
about me
  • 70s – BSEE, coded in Fortran and APL.
  • 80s – went to CS graduate school at UNC-CH, ABD. Started a building business.
  • 90s – started a libertarian think tank.
  • Y2K – returned to programming to make a living, focus on Java and Internet programming.
slide3

The Familiar Catch-22

  • You can’t get a job without experience.
  • You can’t get experience without a job.
  • So I needed a project to work on at home.
why this project
Why this Project?
  • Spam is an interesting problem to me. It is a “tragedy of the commons”, an example of lawless behavior in a public space.
  • An email service could eliminate spam in a novel way – by charging unlisted senders.
  • Along the way I would learn about programming at the level of sockets and network protocols – where the problem of Internet lawlessness seems to start.
minimal specification to offer an email service which eliminates incoming spam
Minimal Specification:to offer an email service whicheliminates incoming spam
  • Customers can manage a list of “accepted” senders
  • All messages from other “unlisted” senders are waylaid. Automatic email is sent back to these senders, explaining the situation, giving them the option to pay for delivery
  • Works with usual email client, such as Mozilla Thunderbird or Outlook Express.
wish list specification postponed
Wish-List Specification:(postponed)
  • Transform payment into bond, with easy way for customer to order refund.
  • Add other ways for unlisted senders to authenticate themselves.
  • Make the service scalable, to handle high volume of traffic.
  • Install defenses against denial of service attacks.
  • And many more features.
this presentation is not
This Presentation Is Not
  • Entirely up to date (by a few years)
  • Comprehensive

Yet It Is

  • Basic material that anyone who programs email will need to know
protocols specification
Protocols Specification
  • I want people to be able to have email accounts with me. What do I have to offer?
  • Protocols required?
  • Services required?
how an email message is transmitted
How an Email message is Transmitted

Sender

Email Client

such as Thunderbird or Outlook Express

simplified SMTP

Recipient’s ISP

Mailserver

or

Mail Transfer Agent

SMTP

Mailserver

or

Mail Transfer Agent

Sender’s ISP

POP3

Email Client

such as Thunderbird or Outlook Express

Recipient

a look at smtp
A look at SMTP
  • RFC (821, 1982), superseded by 2821, 2001
  • client opens a socket to port 25 on the SMTP server
  • SMTP server identifies itself
smtp in action telnet from dos
SMTP in Actiontelnet from DOS

C:\>telnet XDomain.com 25

220 gork.XDomain.com ESMTP Postfix

helo mailscreen.net

250 gork.XDomain.com

mail from:<sue@yrox.net>

250 Ok

rcpt to:<fred@XDomain.com>

250 Ok

data

354 End data with <CR><LF>.<CR><LF>

Here is a message to you.

.

250 Ok: queued as 63235619F2ED

quit

221 Bye

Connection to host lost.

content of smtp data rfc 822 2822
Content of SMTP dataRFC (822) 2822

data

354 End data with <CR><LF>.<CR><LF>

Received: from BILL ([73.175.198.24]) by vm024.mailsvcs.com

for rHammer@mailscreen.net; Wed, 03 Jan 2007 07:07:50 -0600 (CST)

Date: Wed, 03 Jan 2007 08:07:49 -0500

From: "Bill Hill" <bHill@Verizon.Net>

Subject: RE: UFO sighting

To: "Richard Hammer" <rHammer@mailscreen.net>

MIME-version: 1.0

Content-type: text/plain; charset=windows-1250

Content-transfer-encoding: 7bit

Hi Rich,

Interesting! Maybe E. Jackson should take a look at it :)

Bill

.

250 Ok: queued as 63235619F2ED

HEADERS

MIME HEADERS

RFC 2045

RFC 2822 headers end with a blank line

smtp rfc 2821 envelope vs rfc 2822 headers

Addresses in SMTP envelope

may differ from

Addresses in headers

SMTP (RFC 2821) envelopevsRFC 2822 headers

mail from:<sue@yrox.net>

250 Ok

rcpt to:<fred@XDomain.com>

250 Ok

data

354 End data with <CR><LF>.<CR><LF>

From: "Bill Hill" <bHill@Verizon.Net>

To: "Richard Hammer" <rHammer@mailscreen.net>

My message to you.

.

250 Ok

quit

221 Bye

socket programming
Socket Programming
  • java.net.Socket
  • java.net.ServerSocket
  • On top of TCP
slide15

public class GetterServer implements Runnable{

boolean gracefulShutdownOrdered;

ServerSocket receiverServerSocket;

Socket receiverConnectionSocket;

public GetterServer() throws Exception{

receiverServerSocket = new ServerSocket(25);

}

public void run(){

while(!gracefulShutdownOrdered){

try{

try{

receiverConnectionSocket = receiverServerSocket.accept();

}catch (IOException e){

if(gracefulShutdownOrdered)

break;

else{

logStackTrace("Exiting GetterServer.", e);

break;

}

}

Getter myMan = new Getter();

myMan.handleConnection(receiverConnectionSocket);

}catch (RuntimeException runt){

logStackTrace(runt);

}

}

}

...

}

Top Level Code

for a

SMTP server

protocol programming
protocol programming
  • We will look at three methods from class Getter, adapted from:

org.apache.james.smtpserver.SMTPHandler

slide17

method handleConnection(Socket)

class Getter{

//adapted from

void handleConnection(Socket socket) {

remoteHost = socket.getInetAddress().getHostName();

remoteIP = socket.getInetAddress().getHostAddress();

in = new BufferedInputStream(socket.getInputStream(), 7024);

smtpCommandLineReader = new CommandLineReader(in);

out = new InternetPrintWriter(socket.getOutputStream(), "Getter says: ",

true);

out.println("220 " + helloName + " ready ");

String clientCommand = null;

try {

boolean getAnotherCommand;

do {

clientCommand = smtpCommandLineReader.readLine();

getAnotherCommand = parseCommand(clientCommand);

} while (getAnotherCommand);

} catch (Exception e) {

if (shutdownOrdered) {

//log normal closing

return;

} else if (e instanceof SocketException) {

//log unusual condition

} else {// might be major

//log error

}

}

closeConnections();

}

slide18

private boolean parseCommand(String command) throws Exception {

if (command == null)

return false;

String argument = null;

String argument1 = null;

command = command.trim();

if (command.indexOf(" ") > 0) {

argument = command.substring(command.indexOf(" ") + 1);

command = command.substring(0, command.indexOf(" "));

if (argument.indexOf(":") > 0) {

argument1 = argument.substring(argument.indexOf(":") + 1);

argument = argument.substring(0, argument.indexOf(":"));

}

}

if (command.equalsIgnoreCase("HELO"))

doHELO(command, argument, argument1);

else if (command.equalsIgnoreCase("EHLO"))

doEHLO(command, argument, argument1);

else if (command.equalsIgnoreCase("MAIL"))

doMAIL(command, argument, argument1);

else if (command.equalsIgnoreCase("RCPT"))

doRCPT(command, argument, argument1);

else if (command.equalsIgnoreCase("AUTH"))

doAUTH(argument);

else if (command.equalsIgnoreCase("DATA")) {

doDATA();

} else if (command.equalsIgnoreCase("QUIT"))

doQUIT(command, argument, argument1);

else

doUnknownCmd(command, argument, argument1);

return !(command.equalsIgnoreCase("QUIT") == true || shutdownOrdered);

}

method parseCommand(String)

slide19

method doMAIL(String, String, String)

private void doMAIL(String command, String argument, String argument1) {

if (!gotEhlo) {

out.println("503 bad sequence of commands, send ehlo or helo first");

return;

}

if (argument == null || !argument.equalsIgnoreCase("FROM") || argument1 == null) {

out.println("501 Usage: MAIL FROM:<sender>");

return;

}

String sender = argument1.trim();

if (sender.length() < 2 || sender.charAt(0) != '<'

|| sender.charAt(sender.length() - 1) != '>') {

out.println("501 Usage: MAIL FROM:<sender>");

return;

}

slide20

//method doMAIL concluded

MailAddress senderAddress = null;

sender = sender.substring(1, sender.length() - 1);

if (sender.length() > 0) {// the usual case

try {

senderAddress = new MailAddress(sender);

} catch (ParseException e) {

out.println("501 Failure to parse InternetAddress from " + "\""

+ argument1.trim() + "\". " + e.getMessage());

return;

}

// screen senders with mailscreen addresses

if (Services.isToLocalDomain(senderAddress)) {

if (authenticatedUser == null) {

out.println("530 Authentication Required");

return;

} else if (!CustomerSet.isMsAddress(senderAddress)) {

out.println("550 No such address");

return;

}

}

} else { // sender.length() == 0, must have been <>

// let senderAddress remain null, as that is our signal of mail from <>

}

setEnvelopeFrom(senderAddress);

out.println("250 Sender " + sender + " OK");

}

mailscreen net as it now runs
Mailscreen.netas it now runs
  • A free offering of the service has been available since March 2004
slide22

SMTP

CLIENT

TOMCAT

JSP

POP3

MAILBOXES

POSTGRES

SMTP

SERVER

Unlisted Sender

T 1.0

PAYPAL

Unlisted Sender Pays

HTTPS

T 0.0

HTTPS URL

T 1.1

TO: JOE@Mailscreen.net

FROM: Unlisted Sender

T 1.2

Payment Required Message

TO: Unlisted Sender

FROM: customer-agent@mailscreen.net

T 1.3

T 0.2

MAILSCREEN

Instant Payment

Notificaton

HTTP POST

Echo Parameteres

"VERIFIED"

T 2.0

T 1.4

T 0.1

TO: JOE@Mailscreen.net

FROM: Unlisted Sender

payment required message
Payment Required Message

Dear Somebody@somewhere.org:

Mailscreen.net has received your email for a Mailscreen customer,

service@mailscreen.net.

Unfortunately, since your email address is not registered as an

accepted sender to this customer, Mailscreen will not deliver your

message to the customer until you make a payment as described in the

next paragraph.

The charge to send this message will be $0.50. To make this payment

click on the link below. It will take you to PayPal.com where you

can make payment, and where you can open a new PayPal account if you

do not already have one. Mailscreen does not accept any other form

of payment at present.

https://www.paypal.com/xclick/business=cashier@mailscreen.net&item_name=accept+email&invoice=060410.204815c5i0&amount=0.50&return=http%3A//mailscreen.net/thanks.jsp

Please make your payment promptly. We cannot promise fulfillment if

your payment comes after four calendar days, because we must limit

the number of messages that we hold at Mailscreen.

If you would like more specific information to identify the message

which we received from you and which we are holding awaiting payment,

here are some identifying headers from that message.

Date: 10 April 2006

From: Somebody@somewhere.org

Received: FROM nc-60-41-56-62.dyn.yahoo.net ([60.41.56.12])

BY mailscreen.net; Mon, 10 Apr 2006 20:48:15 -0400 (EDT)

Sincerely,

Mailscreen.net

slide24

Instant Payment Notification, in the JSP that PayPal calls

<%

Enumeration en = request.getParameterNames();

StringBuffer replyBuf = new StringBuffer("cmd=_notify-validate");

while(en.hasMoreElements()){

String paramName = (String)en.nextElement();

String paramValue = request.getParameter(paramName);

replyBuf.append("&").append(paramName).append("=")

.append(URLEncoder.encode(paramValue,"UTF-8"));

}

// post back to PayPal system to validate

URL url = new URL("http://www.paypal.com/cgi-bin/webscr");

URLConnection uc = url.openConnection();

uc.setDoOutput(true);

uc.setRequestProperty("Content-Type","application/x-www-form-urlencoded");

PrintWriter pw = new PrintWriter(uc.getOutputStream());

pw.println(replyBuf.toString());

pw.close();

BufferedReader in = new BufferedReader(

new InputStreamReader(uc.getInputStream()));

String res = in.readLine();

in.close();

if (!"VERIFIED".equals(res)){

log("Payment failed. res not VERIFIED but "+ res);

return;

}

jsp control consoles
JSP Control Consoles
  • Administrator
  • Customer
problem how can calls from web app reach the objects in the mailserver

PROBLEMHow can calls from Web Appreach the objects in the mailserver?

SOLUTION CHOSEN

Run Web App and Mailserver in

the same JVM

slide27

Starting the Mailserver

It Is a

Servlet

public class StartServlet extends HttpServlet {

public void init(){

ServerManager.start();

}

public void destroy(){

ServerManager.myOneServerManager.startGracefulShutdown();

}

}

slide28

mailscreen web.xml file

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE web-app PUBLIC "-//Sun Microsystems, Inc.//DTD Web Application

2.3//EN" "http://java.sun.com/dtd/web-app_2_3.dtd">

<web-app>

<servlet>

<servlet-name>do it damn it</servlet-name>

<servlet-class>common.StartServlet</servlet-class>

<load-on-startup>3</load-on-startup>

</servlet>

<!-- security for regular customers -->

<security-constraint>

<web-resource-collection>

<web-resource-name>customerResource</web-resource-name>

<url-pattern>/customer/*</url-pattern>

</web-resource-collection>

<auth-constraint>

<role-name>customer</role-name>

</auth-constraint>

<user-data-constraint>

<transport-guarantee>CONFIDENTIAL</transport-guarantee>

</user-data-constraint>

</security-constraint>

...

slide29

Tomcat server.xml file

<Server port="8005" shutdown="SHUTDOWN">

<Service name="Catalina">

<Connector port="80"

maxThreads="15" minSpareThreads="1" maxSpareThreads="2"

enableLookups="false" redirectPort="443" acceptCount="6"

connectionTimeout="20000" disableUploadTimeout="true" />

<Connector port="443"

maxThreads="10" minSpareThreads="1" maxSpareThreads="2"

enableLookups="false" disableUploadTimeout="true"

acceptCount="5" scheme="https" secure="true"

clientAuth="false" sslProtocol="TLS" />

<Engine name="Catalina" defaultHost="localhost">

<Host name="mailscreen.net" appBase="/usr/website/WebRoot"

unpackWARs="true" autoDeploy="true">

<Alias>www.mailscreen.net</Alias>

<Valve className="org.apache.catalina.valves.AccessLogValve"

directory="msLogs" prefix="access." suffix=".log"

pattern="common" resolveHosts="false"/>

<Realm className="org.apache.catalina.realm.JDBCRealm" debug="99"

driverName="org.postgresql.Driver"

connectionURL="jdbc:postgresql://localhost:3246/pqlms"

connectionName="mailData" connectionPassword="e3$fe4uii0p"

userTable="uLogin" userNameCol="uName"

userCredCol="cred"

userRoleTable="uRole" roleNameCol="power" />

<Context path="" docBase="." />

</Host>

</Engine>

</Service>

</Server>

complexity of smtp sending classification draft
Complexity of SMTP sending, Classification draft

M4 - a message as received in SMTP, with potentially:

  • many domains among the recipients
  • many delivery attempts (try or retry) necessary for each domain
  • many servers (IP addresses) to attempt during each delivery attempt
  • many recipient addresses to attempt when a connection to a server succeeds

M3 - part of an M4, with recipients at only one domain, but with potentially:

  • many delivery attempts (try or retry) necessary for each domain
  • many servers (IP addresses) to attempt during each delivery attempt
  • many recipient addresses to attempt when a connection to a server succeeds

M2 - one of attempts (try or retry) to deliver a M3, but with potentially:

  • many servers (IP addresses) to attempt during each delivery attempt
  • many recipient addresses to attempt when a connection to a server succeeds

M1 - part of an M2, the attempt to deliver an M3 to one server (IP address), but with potentially:

  • many recipient addresses to attempt in this connection

M0 - part of an M1, the attempt to deliver an M3 to one user address at one server (IP address) during one try or retry attempt

mailscreen key classes

(abstract)

M4

List: envelopeTo

MailAddress:

envelopeFrom

String: id

File: dataFile

MailscreenKey Classes

1..*

1

DistinctDataMessage DDM

String: ddmId, status

boolean: disposable

(abstract)

(interface)

MessageToOneRemoteDomain

MORD

List: recipients

String: domainName

Remote

Recipient

RR

MailAddress:

address

String: status

MessageToMsRecipient

MSR

void deliverTermsMet()

void stopWaiting()

1..*

1

MessageToNonMs

MsMailBoxEntry

void writeTo(OutputStream)

features

ForwardedMessage

toCustomer

features

1

1

1

1

MsMessageFeatures

Customer: customer

MailHeaders: headers

delivery status notification
Delivery Status Notification

This notification pertains to a message you sent. You may identify

that message by these headers taken from it:

To: susie@foggybottom.net

Subject: coming through

Date: Sun, 28 Mar 2004 20:22:21 -0500

This message could not be delivered to one or more recipients. A

permanent failure was encountered while attempting to deliver to:

susie@foggybottom.net

No further action will be taken here on your message.

lessons about spam
Lessons about Spam
  • Spam is not bad enough to drive people to the tactless solution offered by Mailscreen.
  • People do not want to risk the possibility that a cherished contact might receive a “payment required” message.
  • The problem is not just technical, because there are many technical solutions.
  • The problem is sociological.
references
References
  • RFC Index http://www.rfc-editor.org/rfc-index.html
  • Delivery Status Notifications RFC 1891, 1894
  • Java Network Programming, 2nd ed., Elliotte Rusty Harold, 2000
  • Programming Internet Email, David Wood, 1999
libraries used
Libraries Used
  • org.xbill.DNS
  • javax.mail

Source Available to You at

  • http://mailscreen.net/jug.zip
  • for one week, until Jan. 22, 2007
thanks to
Thanks To
  • Sun Microsystems, for Java, the Java API
  • Apache James project
  • Ralph Cook
  • IETF email list, ietf-smtp@imc.org
  • Triangle Java Users Group