next »

[jelv1]

That depends upon being able to get a complete list of the emails in the first place - with so many that is not possible by any means.

[Ultra]

"not possible" or "not easy"...   I think it very possible, but not by "normal" means using existing mail clients.

For example, some years ago, while working for a small transport consultancy, I was introduced to MegaBasic (and yes, I know many *hate* BASIC, the learning language which some think is only suitable to those of school age), however, it allowed one to compile and was very, very, good in a number of areas, and nothing like the various 'dialects' I had used before (with my earlier 10+ years in programming, I could think of around a dozen different versions of BASIC, from different software houses, not the v1, v2, v3 type!)

I'd expect it to be quite possible, (and even quite easy) though time consuming, to read the file as a sequential series of characters (that's presumably how it is at PN) and split it into however many thousands of separate messages, while also building lists of 'from' 'to' 'subject' plus message number and size of each message.  Then only the mail from wanted senders or without spam subjects might be considered for viewing.

I'd offer to do the programming myself, for free, but have a number of other, quite urgent, committments on the go at present.  However, reading files and creating others with processing in the middle is what I've had experience of for 20+ years.  It was easy to make a 45,000 page website (of static info, each page linked to 'parent' and 'offspring' pages) for a client, with a little programming time/testing.

Pages were live before they decided on some other changes (and the search engines had already processed these pages...  we knew that from the web logs and enquiries from one of HMG's defence departments...  I still remember part of the e-mail address being @...dra.hmg.gb - not the normal  .*.uk type, and the web logs showed some interesting host names too and we pulled the site content before everyone and his dog started looking at the data...  it was not 'sensitive' - just trade stats (the same transport firm, wanted this done fast for someone abroad to quickly examine some sample data).

Surely for PN the task is to copy this mail and zap the original, such that a new set of mail can start building up (perhaps with some filtering, and if Penny needs it forwarded to a Google Mail account, with 2 GB capacity, and some degree of automated spam filtering at their end, I'd be happy to send her an 'invite' - just get a free fastmail.net or yahoo.co.uk mail address Penny, so I know it won't go anywhere near PN mail or be subject to getting lost/delayed!)

[Penny]

Appreciate the additional info.  Have to say I'm not over the bronchitis yet and find myself faced with a deluge of (other) seemingly-critical stuff that all seems to need to be dealt with simultaneously.

I'll wait to see what Networks come up with on Monday;  the one thing that does strike me as essential at this point is to know how to set up a "blackhole future default e-mails" facility on the e-mail settings page where I initially set up the new mailboxes.

If by some miracle Networks were able to re-queue the mail, such a blackhole facility would entirely cure the past/present problem whilst extricating the important stuff via the specific mailboxes.  Appreciate re-queuing might not be possible   but even if Networks (1) export the 100-150,000 as a file and/or (2) clear the existing default mailbox, I am going to be facing a progressive repeat of the overload, assuming mail is still coming in at 5000/day, unless I have got the blackhole redirect already in place on the default mail.

I have about seven (I think) identified-specific-mailboxes now in place.  Could someone please explain (if poss in a way even I can't misunderstand) how to set up the blackhole for all-other-mail, and I'll put that in place? Even if I'm not able to do anything else right now towards sorting out the mail issues, because of all the other things going on here, I trust that would at least be one small step in the right direction

Thanks,

Penny.

[Ultra]

Yes, you certainly need to be in a position where it doesn't start happening again immediately the mail backlog has been cleared/moved from the current storage...

I'll leave this to someone who is much more familar with how to get the 'default' to be switched from your regular (account name) mailbox to be the blackhole destination. 

(Although I suggested the blackhole facility a while back, I tended to keep all my mail separate from PN, as I've had my own domains {complete with spam problems} since '95/'96, and as a result, never got as far as using the PN mail service to the degree others have.)

Sorry to read about the bronchitis, Penny.  Hope you get over it ASAP.  My sister went down with a chest infection at least a month before the end of term (she's head of a junior school) and had a couple of weeks off, switched anti-biotics and was back in for a bit, but even now, many weeks on, is still not 100%, unfortunately.

[Penny]

I'll leave this to someone who is much more familar with how to get the 'default' to be switched from your regular (account name) mailbox to be the blackhole destination.

Turns out it is Support who need to be asked to implement that!  so I have raised a ticket requesting them to blackhole everything save the specific-mailboxes.  That should hopefully sort out the "future" stuff even if not the present default-mailbox contents.

Just sent some test messages to the existing specific-mailboxes; results as follows:
(1) penny2006 authenticated and downloaded the e-mail.
(2) penny2007 authenticated but didn't download any mail.
(3) pokemon2006 wouldn't authenticate.
(4) pokemon2007 authenticated but didn't download any mail.
(5) michael2007 authenticated but didn't download any mail.
(6) postmaster (redirect on mainmail - thanks to jelv for finally making me understand redirects!) worked fine.

So out of six mailboxes and one redirect, one mailbox wouldn't authenticate (pokemon2006) and three mailboxes (penny2007, pokemon2007 and michael2007) authenticated but didn't contain the test e-mails.

On a re-try of the last three, michael2007, penny2007 and pokemon2007 wouldn't authenticate at all.

Assume the above four issues are connected with the current mail problems 

Not looking good 

Penny.

[Ultra]

I would have been suggesting use of Gmail (where POP collection is, I understand, possible) but you have the domain with PN as well, so still at the mercy of the PN mail service to accept the messages and then forward them on, rather than them be forwarded (from some separate hosting service to Gmail).  So, must just cross my fingers for you.

Glad you're up to speed on the redirections, and that PN support has the matter of the blackholing in hand for the 'catch all' option.  I'm even gladder that I can do that (for the domains I manage, and my own) without needing to ask anyone - just change a destination address to :fail: (to give a 550 error) or to :blackhole: (to make the mail disappear without trace)

[Penny]

Glad you're up to speed on the redirections, and that PN support has the matter of the blackholing in hand for the 'catch all' option.

Not entirely sure that the new specific-mailboxes are actually working fully (see Email still Fubared thread) but major progress now re the overload on the default mailbox (ticket progs follow):

Dear Penny,
After looking at this and talking to Mark, I have moved your default mailbox to a temporary location and created a new, empty one. This will allow you to log in to the mailbox and view new mail. This does however leave us with the problem of what to do with the large amount of old mail.

If you could give us details of the email you would like us to retrieve for example a list of addresses the email may have been sent from then we can move this into your mailbox for your perusal.

Regards,
Ben Brown

.....
Thank you Ben   spirits lifted somewhat;  I have been finding all this a little difficult to deal with :/

Firstly excellent news about the cleared mailbox.  I've just re-enabled the default-mailbox call-up in OE and got 147 messages through - only about an hour's worth but maybe the 5000-a-day rate has dropped somewhat in the interim

2) would it be possible for you guys to check the specific-mailboxes on the account are working?  test e-mails to pokemon2007, penny2007 and michael2007 (sent 7th Jan and 9th Jan) haven't got through (though I have just received a single e-mail from a friend addressed to penny2007, first receipt of all on that address).

3) if the specific-mailboxes are confirmed to be working, would you (please) be able to set up the blackhole-any-non-specific-mailbox-stuff facility, for the rest of the new mail, now the default mailbox is working again?  I'll set up more specific-mailboxes in due course, probably, but it would just reduce any overload facility for the time being and ensure that important/urgent stuff can still get through to here.

4) re the "old mail", the only really-essential items are likely to be to penny2006, pokemon2006 and michael2006 (all at the happychild address), as well as penny2007, pokemon2007 and michael2007 if any of those have failed to make it into the specific-mailboxes with the recent mail problems.  If it might be possible to extricate those six addresses, the rest can be disposed of, realistically.

Thank you *so* much for sorting things out.

Regards, Penny.

Having mail "working" again is really quite strange! but very reassuring.  Hopefully the specific-mailboxes will all operate properly soon, and I can then get the remaining ones set up.  A huge thank-you to all who've offered help and advice on this thread

Penny.

[Penny]

Just a brief update on all this ....

Endless thanks to Ben Brown who had not only cleared the default mailbox so it could be used, but then systematically went through the removed 100,000+ emails from the preceding four weeks to extricate the genuine stuff, and forwarded it here.  Virtual knighthood on its way, Ben

All the test e-mails did eventually arrive, and the specific-mailboxes all seem to be working okay;  meanwhile since yesterday evening, overnight, the ongoing 5000-ish a day inbound e-mails have dwindled to a tiny trickle.  The black hole is now in place.

My thanks to everyone who has played a part in sorting all this out.  It's like an oasis of calm here right now and the restored e-mails have filled in the gaps in the jigsaw, so I can progress things properly with the site again.

Likely to be a tough year ahead but I'm feeling decidedly more optimistic about it right now


Penny.

[MauriceB]

I'm really glad to hear this Penny.  It must have been extremely frustrating (not that it came over as such ) to not be able to see your mail, nor to do anything much about it!  A powerful warning to others on the perils of ignoring SPAM protection.

There is also something to say about the approach:  Reasoned problem statement and follow up data without 'throwing the toys out of the pram' seems to have paid dividends .  Yes it has taken time, but the team got there.

I'll add a Bar to Ben's virtual Medal for going the extra mile.

Maurice

[dhookham]

I'll add a Bar to Ben's virtual Medal for going the extra mile.

I'll raise a glass of a particularly good malt in Ben's direction 

[Ultra]

Glad to see things are going better for you, Penny, and hats off to Ben in particular, in helping sort this out.


Just goes to show how much of a pain spammers can be, and a shame that some UK, US, and especially Far Eastern (!) government and parliamentary/decision making mail servers have not also been affected, so they'd do more in respect of this problem.

In the US, it still seems to be a "freedom of speech" issue.  Here, I'm not sure much has been / can be (esp if sources are overseas, as seems often to be the case).

One hosting firm I used has made the 'catch all' (default) to be a 'fail' - so senders get a 550 error message and mail to unknown users is not accepted. 

Not sure if they have told users they should have a 'postmaster@' account (probably, but may be hidden in the depths of their FAQ documents), but clearly they have had problems with customer domain default mailboxes getting quickly filled with junk, and giving them headaches too.  They run several hundred servers, with both individual and reseller accounts spread over them, so get lots of mail!

[lmartin]

I've passed on your thanks to Ben   Always good to hear positive feedback.

[Penny]

I've passed on your thanks to Ben

Thank you Liam

[billbow]

Hi Penny

A happy ending. ahhhh. 

Bill

[Penny]

Hi Penny
A happy ending. ahhhh. 
Bill

Hi Bill,
Indeed it is   I am finding it *so* much easier to progress site stuff now because mail-sifting takes so much less time.  Having said that the house is surrounded by broken fence panels after the storms yesterday, and my mum has taken a turn for the worse again, so the "calmness" factor has dissipated somewhat - but dealing with the rest of what is going on here is infinitely easier now that the e-mail situation is fully, finally, sorted out.

Finally got to see the supervisor at the Epping Post Office today and yes they really do seem to have mislaid all the December PO Box mail ... weird parallel.  These things happen, I guess - just reassured that the site at least is becoming easier to keep on track; the rest of my life I will seemingly have to do in stages

Regards,

Penny.