Satphone Email Compression

[John Hayward]

Hi All,

I just got an Iridium phone with the data kit for the transpac (and beyond). I know there are compression programs/services to reduce the email send/receive times, but with the ones I have found the service costs more than you would save from the reduced airtime due to the compression unless you do a LOT.... Has anybody found a program/service that isn't more expensive than the airtime it saves?

Thanks,
John

 

[sleddog]

Hi All,

I just got an Iridium phone with the data kit for the transpac (and beyond). I know there are compression programs/services to reduce the email send/receive times, but with the ones I have found the service costs more than you would save from the reduced airtime due to the compression unless you do a LOT.... Has anybody found a program/service that isn't more expensive than the airtime it saves?

Thanks,
John

An experienced offshore racing friend attempts to answer your question:

The Iridium Phone and data kit comes with software "Iridium Direct
Internet 2.0.2" (at no additional cost for the software). This software
can provide compression for e-mail and for graphics. As to e-mail,
ordinary text messages require very little time to download so
compression does not add much. It is only e-mail that includes an
attachment that is a problem because attachments such as pictures can be
very large in size. Even with compression these large attachments take
substantial time to download. As a general rule, with a narrow-band
system such as Iridium, one should not attempt to download attachments.
Some of the software e.g. Outlook Express, can be set up to download
only the text e-mail and to exclude attachments. I would recommend
that the e-mail software be set up so as to exclude attachments in order
to avoid downloading very large files.

Downloading files via Sailmail and Winlink using Iridium is a somewhat
different matter. The Sailmail system is set up so that it excludes
all attachments with the exception of grib files (or files that have
been labeled as grib files with the "grb" extension at the end of the
name. Grib files are already substantially compressed so that
additional compression is not very useful because such additional
compression does not reduce the size of the grib file substantially.
The Sailmail site states that Sailmail provides compression (perhaps
only with Pactor III?) that apparently obtains about as much compression
as is available so that additional attempts at compression provide very
little reduction. The bottom line is that it is my impression that
other services that "provide compression" do not actually provide any
significant advantage relative to the stuff that is already available.
The Winlink system allows you to set the maximum size of any attachment
that will be transmitted to you.


With either Sailmail or Winlink, using the "telnet" protocol will often
reduce the transmission times significantly and I recommend using
"telnet". However, I have experienced occasional problems using telnet
for downloading large data files, apparently because the Iridium
satellite system has some problems with handing off data transmissions
between satellites. As a consequence, downloads of data files should be
attempted only when there is a "solid" satellite connection. I have
also sometimes experienced a problem with a download via Sailmail and
Iridium that had failed or "stalled". When I attempted to re-download
the file, the system rejected my request. I'm told that Sailmail is
"hung up" on the previous "stalled" request and will "clear itself"
after 5 or 10 minutes, and that I should wait 5 or 10 minutes before
attempting to re-download the file. I think Jim at Sailmail is working
on this problem. I have not attempted to download grib files using the
default Iridium protocol so I do not know if the "stalling" problem can
be avoided by not using "telnet".

Joe (KC6I)

 

[seabird51]

E-mail compression at sea

Hi John
Joe of Sleddog gave a very thorough assessment of email compression using Sailmail etc via SSB and Pactor modem. If you want a system not SSB related or have, as I do, a good older ICOM SSB that can not be set-up for email, then the Iridium is a good option.
I used the basic Iridium phone without data for the 2002 SHTP. In 2006 I had a data-port and the basic Iridium system to download gribs. The gribs were free from 2 sources. They were very useful but expensive in terms of airtime. I seem to recall that they took about 10+ minutes to download. Add this to the regular emails, which were also pretty slow, and phone conversations and I ran up a $2000 bill for airtime over a 2 month period.
This year for going to Mexico I added the Global Marine Maxgate (X-gate for PC's) compression system. The same 4 day grid downloads in 3-4 minutes and text emails, a couple going each way, take about 2 minutes. With no mail coming or going, I can check for messages in less than a minute.
In the end, for a lot of email, daily gribs and an occasional very small photo, that software and system is cost effective.
Additionally, the tech support at GMN (Luis Soltero) is amazing so you never have to deal with Iridium people.
Lou
Seabird

 

[lsoltero]

Let me start my reply with a question.

"Have you ever tried to do e-mail over Iridium without using an e-mail system designed to work over the satellite network?"

What you will find is that it is not only expensive to do so but also very frustrating.

Forget compression for the moment and lets just focus on one feature most programs support. Big mail filtering. What would happen if some one sent you an e-mail with a photo (say 1Mb in size) to your earth link account. When you tried to download that into our e-mail client you would find that until that one large mesg were downloaded your mail stream would halt. you would also find that at $1.50 per minute it would take 69 minutes to download that one message. So lets say that you had no choice and that you started the download of this large mesg and tht 68.5 minutes into the transfer the satellite connection were dropped. On reconnect you would find that you would have to download that one mesg all over again.... A very expensive proposition indeed.

There are other things besides mid file restart and big mail handling that compression systems provide. One important one is protocol latency removal. Over iridium it takes 3 seconds to receive a reply from a server on the internet to a request. When sending mail using the standard SMTP protocol (this is what outlook express and other e-mail clients use) there are 5 line turn arounds required of static overhead plus one line turn around per recipient. This means that if you were to send a small e-mail to one user address 18 seconds (15+3) would be lost in protocol overhead before a single byte of data is transfered. Send that same e-mail to 10 individuals and you will find that 45 seconds are lost before a single byte of data is transmitted. At $1.50 these latencies get expensive.

Throw in pipelining, compression, duplex operation, spam and virus filtering and these compression based e-mail services start looking like a bargain.

So you say you will use the web to connect to hotmail.... Think again... First most webmail systems use secure https:// websites. This means that the data stream is randomized by encryption. By definition these streams can not be compressed. At 2400 baud (on a good day) you will find that about 1Mb of data has to be transfered to get to your inbox in hotmail before you can start reading your mail. So... might mean 30 minutes of air time just to get to a point where you start reading mail. Then what happens if you loose your connection? Yup.... You start all over again.

I cant speak about other vendors but our XGate product sells for about $20 per month. That is 13 minutes of Iridium air time. You will find that you will save considerably more than $20 per month in air time by using XGate.
Without optimization software (note that I didn't say compression based software since there is a lot more to this than just compression), my prediction is that, you will dump the Iridium phone in the drink and give up on e-mail altogether.

On the issue of Direct Internet software provided by Iridium for free... This is a system that was developed for land based wireless solutions such as cell phones. note that compression systems for cell phones and other land based slower wireless modes mostly do compression and do nothing about all the other factors. The reason for this is that the latency of the line turn around is negligible. Land based wireless is more reliable and cheaper. not to mention that the connections are much faster than satellite. So as mentioned above you only get compression which for small file transfers will give you marginal if any performance improvement. So... with direct internet you get compression but you do NOT get protocol overhead removal, duplex operation, pipelining, block compression, mid file restart, big mail handling, or spam and virus filtering. And to make things worse once you are connected to the internet it takes the software about 20 seconds to negotiate its connection to the compression server so in essence you use 40 seconds of air time (20 to negotiate ppp and 20 to bring up the connection to the server) before you send a single byte of data. Bottom line... Most people do not find Direct Internet very helpful.


BTW... XGate does not have the issues that airmail in terminal mode has with iridium as discussed in a previous post. The compression scheme that XGate uses is totally different than airmail and it has been optimized for Iridium. XGate is one of the very few Iridium certified e-mail systems out there. It is certainly the only one for Mac OS X and Linux. Large file downloads are not an issue with XGate. That is one of its strengths. Copies of the Iridium certification letters can be found on the awards page of our website for those interested.


Take care.

--luis


Luis Soltero, Ph.D., MCS
Director of Software Development, CTO
Global Marine Networks, LLC
StarPilot, LLC
Tel: 865-379-8723
Fax: 865-681-5017
E-Mail: [email protected]
Web: http://www.globalmarinenet.net
Web: http://www.starpilotllc.com

 

[John Hayward]

Thanks to all for the very detailed responses. I have been doing Airmail for over a decade, since the Pactor 1 days but had forgotten or never known about the Telnet capability. Airmail via HF will be my primary means of communication for the race, but I wanted the Iridium for backup both for voice and data. Luis, thanks for your very informative response. I never had any intention of using the satellite phone to connect to my regular email servers. I have had all the problems you mention when I do it via my cellphone and don't wish to do it at $1.50 per minute. I am sure the benefits you mention are worth every penny if the satellite phone is your primary means of communication. I will have 2 HF radios and 2 separate antenna systems, so the satellite phone for data is truly a "backup." Having said that, I will try it using Airmails telnet function well before the race and if this is not acceptable will be giving you a call. I hope to only use the satellite phone to check in with the "better half" occasionally if conditions on HF don't allow "chatting" (we are both Extra Class Amateurs).

Thanks again to all.

John Hayward
Dream Chaser, Valiant 40

 

[Jeff Lebesch]

Thought I would throw my 2 cents in on this subject. Last year I purchased OCENS (www.ocens.com) email and weather file programs, which are optimized for the bandwidth-impared. On two occasions, a charter in Belize and the LongPac, I rented an Iridium and was very pleased with the setup. The software was a bit challenging to install, but the technical support was excellent and I was running quickly. The email program works superbly. I could send and receive a few emails with just a few minutes of air time. The phone worked inside my cabin without an external antenna. If a person did not want to spend a little extra for the weather program (downloads GRIBs) you can get the same information via email for free from Saildocs (see www.saildocs.com for info). A GRIB reader will be required.

For big email users, a ham or SSB and Pactor modem might be cheaper, but the Iridium combined with an optimized email program, is easy to get going, to use, and for moderate usage is a great option. It will be my primary for the TransPac.

Jeff Lebesch
Hecla

 

[AlanH]

I'm planning on using my "other" work e-mail address, the one associated with my specific job, rather than my general university account. Nobody has this address, I never (EVER) get mail at it and in fact anything that DOES come in is auto-forwarded to my general university address.

I'll test it by turning off the forwarding feature in a few weeks and seeing if any mail at all comes in, but I doubt it. This is a POP server and an SMTP server all on the same box. I get no spam there and the only people who will have this address are those whom I give it to...TransPac friends who know not to send me attachments.

I'm going to rent an Iridium phone for a month. It will arrive 4-5 days before the race so I can spend a couple of evenings setting up the software and sending a couple of test messages. I plan to fire up the Iridium phone and dial their data servicing gateway, and send SMTP mail...ONE message, which will probably be pretty long, long 400 words, about every three days during the race. OK, so that's five times during the SHTP. I assume I'll be downloading 3-4-5-6 messages from friends each time. I'm guessing that this will take about 5-10 minutes. Let's say it's ten minutes.

Ten minutes at $1.50 a minute is fifteen bucks. if I do that five times, I just spent $75 and was getting data for 50 minutes..

If I want to set up X-Gate, I have to pay an up-front, one-time fee of $59, and then $20 a month for a minimum of three months. That's $119. Let's say that X-Gate cuts my airtime in half. So instead of having the phone on for 50 minutes, it's 25 minutes.

The only time the X-Gate thing might pay off is when I download GRIB files. I don't want to do this a lot, just probably 2 times during the race....like on days 3 and 6, when I'm making big decisions on how far south to go. After that I figure I just point the bow at Kauai and GO. If it takes 10 minutes to download those GRIB files on those two days without X-Gate, at $1.50 a minute, then that's 20 minutes and each of those files cost me $15....or $30 for the two of them.

Using X-Gate I'm online for 5 minutes each to get those files, ten minutes total, and they cost me $15 for the two.

Let's add up the total cost for sending/receiving text-only e-mail five times during the trip and downloading two GRIB files.

WITH X-Gate

Set-up fee...$59
3-month minimum @ $20 a month... $60
airtime: 35 minutes @ $1.50 a minute... $53

TOTAL: $172

WITHOUT X-Gate

airtime: 70 minutes @ $1.50 a minute... $105

TOTAL: $105

Even if I downloaded another GRIB file on day 8 or 9 that would only add another $15 to the cost...$120 total.

CONCLUSION: for someone who is going to rent an Iridium phone, use 3-mail occasionally during the race and download GRIB files just a few times, signing up for X-Gate doesn't make sense.

For someone who spends months on the boat and checks e-mail twice a day and gets a mess of e-mail with attachments, X-Gate makes a LOT of sense.

 

[AlanH]

In fact, for anyone running LINUX or OS X or Windows XP (I'm running Win 2003 server at home 'cause I'm a geek) on a dedicated line that doesn't time out like DSL or a cable modem, you can set up your own e-mail server. There are freeware e-mail server programs out there. Here's one for PC.

http://macallan.club.fr/MMS/index.html

There used to be a free version of SL mail from Seattle Labs that worked really well, though their previous products had some problems when used for a big enterprise.

It would cost all of $20 to register, say....www.mySHTPmail.net for a year. That's just an example. You could pick whatever domain name appealed to you as long as it wasn't already taken.

So once you had the mail server running and your IP address registered with that Domain, well...make yourself an e-mail account.

[email protected]

Don't tell anybody but your wife/gf and your five best buddies what it is, and use it to send/receive spam-free and attachment free e-mail during the race.

You get away from all the flippin overhead server chit-chat and attached images and advertising that you have to deal with, with hotmail or yahoo mail.

It's easy. It's REALLY easy if there's someone at home who knows enough about the computer to re-start it in case of a power failure. If you load up the mail server as a service on Win2003 server they don't even have to remember to start the program, they just turn the computer on. I can't do it because Joan is coming to Kauai, and we can't leave the computer on for the ten days she'll be gone. But aside from that, it's a no-brainer as long as you're on DSL or a cable modem at home.

Running a totally secure e-mail service for a big company is no joke. That's work. Running a short-term e-mail server for three months, with a dozen users is easy....matter of fact, I ought to offer this as a service to future SHTP'ers, when I can be around to actually make sure the machine is still running.

 

[John Hayward]

For Jeff and Alan and everybody else out there that's interested,

Since my initial post, I have tried several times to connect to Airmail via telnet with my Iridium phone and it works great. I have sent and received several emails and it's almost instantaneous. Airmail converts HTML to plain text and compresses it and eliminates attachments if you tell it to. Since I plan to use it over HF as primary, being able to use it with the Satellite phone too is perfect.

If you are planning to use a satellite phone as primary communication and are a ham operator, you might consider Airmail. You just need to be a ham to get set up and it's free and you never need to connect via HF. If you are not a ham and don't want to take the written test (code is no longer required) then there is Sailmail, but the yearly subscription is $200 or $250, so for just the race, probably not worth it.

Only a few people have my Ham Radio email address and they are already used to short messages and not just hitting the reply button and copying my original message.

The thing that really messes up any of these systems is if somebody "winks out" and sends you a 5 MB attachment and the system isn't set to delete the attachment, you are effectively shut down for the remainder of the trip because you can't download it.

I haven't requested and downloaded a grib file over the satellite phone yet, but at ~ 6 KB, it shouldn't be too bad, time wise, less than 2 minutes.

For those that are using Satellite Phone as primary, how do you plan to charge it????

John H
Dream Chaser

 

[Alchera]

Downloading large gribs on Iridium with and without XGate

I recently went through the exercise of determining the best way to download large grib files using an Iridium satellite connection, and thought folks might be interested in the results. As a test case, I used a 14 day, 12 hour 1by1 grid grib from Saildocs. This results in a file size of 64,512 Kbytes.

A file of this size can't be received using SailMail, since it exceeds the maximum grib file size they will download to you. There is a workaround, though the SailMail folks probably wouldn't appreciate it. You can break your grib file requests into smaller separate individual requests, and then use a free utility you can download from the www.globalmarinenet.net site to patch the files back together into one large one. But if you don't want the hassle of doing that, you need to use something besides SailMail.

My recent experience with Iridium wasn't all that great initially. The first time I tried to download the file the connection was dropped after about 3 minutes and 10Kb of downloading. This was just using Microsoft Outlook and a standard dialup network to the Iridium phone. No special compression software.

I reconnected, and Outlook began downloading the file again from the beginning. This time it downloaded until it was 2Kbytes from being complete, then it dropped the connection again. Over 10 minutes of airtime totally wasted at this point, and still no file.

Two years ago this worked a lot better. Perhaps Iridium has gotten a little flakey in that time. It's also possible that my ISP (GMail) just dropped the connection because the transfer speed was so slow. Sometimes ISPs will do that if a single mail transfer seems to be taking too long.

At that point I decided to install XGate. With XGate, the same file downloaded completely in 4:19 minutes. For the next test, I started downloading another grib file of identical size, but then manually forced the connection to drop after it was 2/3 done, about 1:35 into the download. (It seemed to be going faster than the first time.) Then I had XGate redial and continue the download, which took another 1:47. It restarted the download from where it left off perfectly.

So I would have to say I'm pretty pleased with XGate. Not only is it significantly quicker to download mail, but the ability to restart a dropped connection and pick up in the middle of a file is a significant advantage that can save a lot of airtime that would otherwise be wasted.

Mark/Alchera

 

[Alchera]

Serial to USB converters and Iridium

On another slightly related topic, I discovered that the Keyspan serial/usb converter I had been using doesn't work on Microsoft XP when connecting to my Iridium phone. The Keyspan unit I was using takes four serial connector inputs and plugs into the computer with one USB connector. The software driver creates four virtual COM ports that you can then reference from your various navigation and communication software. So I have one port for NMEA data (wind, GPS, heading, etc.), one for the SSB pactor modem, one for the Iridium, and one for the AIS engine.

The Keyspan worked fine on Windows 2000. Then, I replaced one of my laptops with one running Windows XP. The Keyspan worked fine for all the ports except the Iridium phone. When I tried to dial the Iridium, the laptop would freeze up and had to be rebooted.

I was able to identify that the Keyspan was the problem. I downloaded their latest drivers and that didn't work. I called their tech support, and they suggested using the Windows 2000 drivers, which also didn't work. So I replaced the Keyspan with a similar 4 port model from Edgeport, which has a good reputation and the software that comes with it allows you to tweak all sorts of parameters, unlike the Keyspan. That solved the problem. So if you are going to convert a serial Iridium connector to USB, I would recommend Edgeport. They also make a single port model if that's all you need.

- Mark

 

[John Hayward]

To Mark and All,
It must have something to do with the 4 port version Mark. I have four of the single port Keyspan adapters and they work great with Windows XP Pro for everything including the Iridium phone. I looked at the price of all the 4 port adapters and compared it with the price of a USB Hub and several adapters and the latter was cheaper and if any one broke or caused a problem, you replaced just that one part and everything else is ok. Great for redundancy. I have run GPS, AIS, Nobletec dongle and Pactor 3 modem simultaneously without problem. I have been using the Keyspan adapters for many years and never had a problem with any serial device.

Wonder if you just had a bad 4 port adapter or is the 4 port is less good at emulation???

My 2 cents

John