View Issue Details

IDProjectCategoryView StatusLast Update
0005105websitewebsitepublic2020-10-28 16:20
ReporterPhil Schaffner 
PriorityhighSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
Product Version 
Target VersionFixed in Version 
Summary0005105: Bogus accounts threaten webite
DescriptionForum moderators have been battling spammers creating bogus accounts by the thousands using automated "bots". The only way moderators currently have to attack the problem is by a laborious process of searching for such accounts and selecting them for deletion. This has been working, although at the cost of considerable time to perform the operations; however, such accounts are currently being created at a rate of thousands per day making deletion of 50 at a time via the web interface a practical impossibility.

Our approach has been to delete all "Inactive" accounts more than 7 days old (these are being created at a rate of about 1 per minute) and "Active" accounts with no posts and either no logins, or with no logins in the last 30 days. The latter are the rapidly growing problem, and more than 40,000 accounts with zero posts created between 7 and 30 days ago currently exist. Account creation at this rate will likely bring the site down if the situation is not dealt with soon.

Proposed approach:

1. Implement some automated way of deleting accounts as described above.
2. Implement captcha or some other mechanism in the account creation process to foil the bots.

Steps To ReproduceMonitor current account creation.
TagsNo tags attached.

Activities

toracat

toracat

2011-09-14 15:47

manager   ~0013259

Yes, I urge all CentOS admins who are in charge of operating the web site to look into this problem.
CiaW

CiaW

2011-09-14 19:36

reporter   ~0013261

I'd suggest Capcha and requiring an email confirmation for account creation. I have both on my site (Drupal -based), but also had to block an I.P. for someone who created multiple accounts in an hour or so. I also have much smaller volume. Would something like fail2ban be an option?

If it were me, I'd shut down new account creation for a (day, 2 days, even a few hours?) until you can get a handle on it and implement something. Then post a notice on the front page with the reason. Thousands per day is pretty ridiculous.
AlanBartlett

AlanBartlett

2011-09-14 21:52

reporter   ~0013264

I am also of the opinion that account creation should be disabled until such time as the attack can be thwarted.

user430

2011-09-14 22:12

  ~0013266

I spent a complete evening trying to implement the captcha solution we have for the "contact" pages into the user setup pages. To no avail. If anybody has any idea on how to do that with our xoops version - be my guest, I'll happily work with you regarding this problem.
CiaW

CiaW

2011-09-14 23:43

reporter   ~0013268

In case it's helpful, since I'm not familiar with xoops - I just did a search for 'xoops new user captcha' and found a thread that mentioned Xoops 2.4.1 and Liaise 1.5 issues; and a user pointed to a solution that worked for him (this is the working link):
http://jordankasper.com/jquery/captcha/
as well as pointing to another link on that message:
http://www.webdesignbeach.com/beachbar/ajax-fancy-captcha-jquery-plugin

Later down the page there's a link for a recaptcha module, so here's the original thread:
http://www.xoops.org/modules/newbb/viewtopic.php?topic_id=70263&forum=28&post_id=321078
kbsingh@karan.org

kbsingh@karan.org

2011-09-15 00:02

administrator   ~0013269

how many posts have there been from these accounts ?

i.e : how many of these accounts have successfully clicked through, activated their accounts ?

if there is a exceptionally low number that answers the above two questions, we might be able to pug in place a script that auto deletes accounts that dont validate within a few hours, or accounts where the initial email does not get delivered.
AlanBartlett

AlanBartlett

2011-09-15 00:44

reporter   ~0013270

A crude but effective method would be to use a cron job.

Currently there are 8174 unactivated accounts.

There have been 68922 accounts created since 2011/8/16, all with zero posts.
Phil Schaffner

Phil Schaffner

2011-09-15 13:25

reporter   ~0013275

One other wrinkle is that a sizable percentage of the obviously bogus Active accounts show a last login date, while others do not. Apparently at least some of the bots are capable of logging in, so just deleting Inactive accounts, and Active accounts with no last login date, more than X days old would help, but is not a complete solution to getting rid of existing undesirable accounts.
Phil Schaffner

Phil Schaffner

2011-09-15 14:30

reporter   ~0013277

Just got done deleting 2117 Inactive accounts more than 7 days old, all created in the last 48 hours since the previous deletion, leaving 6442 less than 7 days old, so the rate on these is currently ~1000/day. The current rate of Active account creation seems to be >2000/day.
kbsingh@karan.org

kbsingh@karan.org

2011-09-17 17:53

administrator   ~0013294

Can someone quantify this :

q: how many of these accounts have successfully clicked through, activated their accounts ?

or is there no easy way to work this out ?

- KB
AlanBartlett

AlanBartlett

2011-09-17 19:36

reporter   ~0013296

Obviously we don't want to play "20 questions" but your question itself is ambiguous. "q: how many of these accounts . . ." What do you define as "these accounts", please?

At present (Saturday September 17th, 2011 @ circa 1935 hours UTC) this is the current situation:

All accounts 96419
Active accounts 88568
Inactive accounts 7848

All suspect/spammer accounts 74361
Active suspect/spammer accounts 66513
Inactive suspect/spammer accounts 7850

(Please note that the figures do not add up precisely due to the fact that new accounts are being adding by the minute.)

It is difficult to obtain these figures via the "klunky" control panel interface. The above is the best we can do . . .
kbsingh@karan.org

kbsingh@karan.org

2011-09-18 00:44

administrator   ~0013299

Alan, since this issue report is about bogus accounts - its a safe bet that we are talking about the bogus accounts.

What is the criteria being used to quantify the 'suspect /spammer accounts', the row you have as 74361 ?
AlanBartlett

AlanBartlett

2011-09-18 02:13

reporter   ~0013301

The current criteria that we are using, following on from a somewhat tedious process of manual examination of a variety of accounts where the "Number of posts is less than one", is an identity constructed of a name+number or number+name (e.g. "peterdotson102" or "1800woodworkin") and the URL field filled with a link to non-CentOS, non-Enterprise Linux or non-computing relevant sites. Such sites are, for example, porn related or offering fake handbags/shoes/clothes or suspect medication or dating services, etc.

In terms of battling against this barrage of bot created accounts we were, up until about 6 weeks ago, manually selecting for deletion: (1) "Only active users", "Last login is more than 28 days ago", "Joined date is more than 28 days ago", "Number of Posts is less than 1" and (2) "Only inactive users", "Joined date is more than 7 days ago". That was tedious but it worked well. The logic being that nobody requires an account to access the site, only to make postings. A genuine user would, once the account has been created, log in and, ultimately make her/his posting. We have been working on the assumption that the initial log-in would occur well within a seven day period and the posting would occur within twenty eight days. Due to the way the Control Panel has been configured, we have had to split the seek-find-delete into two separate steps.

(An aside -- One of the biggest problems with the Control Panel is that it will only allow 50 records (accounts) to be dealt with at a time.)

If there is no immediate solution available, please consider creating two cron jobs, search criteria as (1) and (2) above, to be executed once a day. By implementing such, the inundation can at least be held at a steady state. If execution once a day is not frequent enough, it can become once every 12 hours . . . right down to once an hour.

Sorry for my sort-of brain-dump (I am only just getting back into normal mode following on from a health issue) but does the above give sufficient detail with regards to the problem?
Phil Schaffner

Phil Schaffner

2011-09-18 03:10

reporter   ~0013302

I fully agree with Alan's statement of the problems, and moderator's approach to dealing with them, except the account names are more general. Many are of the forms:

FirstnameLastname - but those never match the email address name.
maryXXXX - "mary" plus arbitrary alpha characters. Occasionally other first names.
XkindXXrows - LOTS of these, and they reappear after deletion.
xxxxxxxxx - semi-arbitrary alpha string, all lower case.
XXXXXXXXX - semi-arbitrary alpha string, all capitals.

Where X may be an arbitrary letter, sometimes a number.
kbsingh@karan.org

kbsingh@karan.org

2011-09-18 22:37

administrator   ~0013305

Alan and Phil, thats a good place to start from. Lets also add in : "accounts not activated within 6 hrs of them being created"

Let me see if I can come up with a way to harvest this info from the backend db to get some sort of a cron job in place. We should still, imho, try to get a captcha included in the signup process in the longer term.
AlanBartlett

AlanBartlett

2011-09-18 23:23

reporter   ~0013306

A good idea. Thanks, KB. I guess that brings us back to the subject of "WebSite V2.0" . . .

I am a believer that if a fast response is necessary, use the biggest non-destructive hammer first and thereby create some "thinking room". In this case, it would be to disable account creation as a temporary measure. ;)
Phil Schaffner

Phil Schaffner

2011-09-19 14:47

reporter   ~0013313

I believe "accounts not activated within 6 hrs of them being created" may be a bit too aggressive, but perhaps our current practice of 7 days is overly conservative. Maybe 3 or 4 days would be reasonable - allowing for creating an account on Friday afternoon and not getting around to activating until the next week.
AlanBartlett

AlanBartlett

2011-09-19 16:21

reporter   ~0013317

Whereas my belief is that something should be done now, if not a week ago last Wednesday, as long as it is "tunable" / adjustable.

Hence my initial suggestion when this was last discussed in the moderator's circle -- "Perhaps it is time for Ralph to disable account creation." :)

user430

2011-09-19 21:14

  ~0013320

Sure I can disable account creation (as to accounts not activated in a certain amount of time: I am rather sure that that all happens automatically and that the window is rather smallish).

I'd really like to have a captcha solution in there (to at least kick out "dumb" bots), but as said: The tutorials I found were supposed to work with the formcaptcha plugin we already have - but they don't do that on the account creation page, but only on the contact page - and I have no idea as to why that is.

Maybe need to revisit this.
Phil Schaffner

Phil Schaffner

2011-09-20 14:27

reporter   ~0013326

I'd say definitely - perhaps with a call for help on -devel. Current status:

9545 inactive accounts, 4193 more than 7 days old.

52988 active accounts with zero posts created between 2011/8/16 and 2011/9/13 and no logins at all, or no logins within the last 7 days.
Phil Schaffner

Phil Schaffner

2011-09-26 19:45

reporter   ~0013401

Current status:

Active Users: 108289 Inactive Users: 13615

8672 inactive accounts more than 7 days old.

65604 active accounts with no posts created more than 7 days ago with no logins in the past 7 days. 86130 total active accounts with no posts.

A quick scan of the 16698 active accounts less than 7 days old shows the vast majority are apparently spambot created.
AlanBartlett

AlanBartlett

2011-09-26 23:49

reporter   ~0013403

Neither Phil nor I are au fait with the intimacies of mysql but, if given access to the raw dbase, we would ensure that deficiency is made good.

However, I suspect that access to the dbase could not be provided to us mere mortals!
AlanBartlett

AlanBartlett

2011-09-28 03:14

reporter   ~0013410

As of the date of this note:

89,170 activated user accounts
14,794 inactivated user account
103,971 total user account

fall into the bogus/spamming category.

Access via the control panel is now very sluggish.
toracat

toracat

2011-09-28 03:53

manager   ~0013411

Reminder sent to: user430

Please give a higher priority to this problem.

user430

2011-09-28 21:45

  ~0013419

For one, I deleted the inactive accounts from the database, which removed 15000 accounts.

I wonder how the spam user accounts can be distinguished - not by mail address, otherwise they wouldn't have been able to activate the accounts.

Maybe remove all accounts with zero posts which are older than two weeks?

Ralph
AlanBartlett

AlanBartlett

2011-09-28 22:59

reporter   ~0013420

Ralph,

Tedious to do so but please review comments 13301, 13302, 13305, 13306 & 13313. That should then give you an idea of what could/should be used as an algorithm for a deletion cron job(s).

If it is possible to disable the ability for *new* account creators to make an account with a link to an external website, that should remove one possible interest to the spambot executors. (Perhaps the ability to insert a website link for existing accounts should be restricted to moderators and above? And while we're at it, ditto for signature blocks?)

If it is possible to adjust the control panel logic such that we can perform operations on more than 50 accounts at a time, it would be a help to us moderators.

Finally for now, as it seems that the initial assignment of this issue to "donavan" was deemed to be inappropriate, perhaps "range" should now have the pleasure? ;)

user430

2011-09-28 23:18

  ~0013421

Yeah, I am sorry, I am still not completely up to par after moving. We cannot restrict url or sig block (but maybe we can take those away from the user creation form) - but I don't think that would stop the automated attacks.

Without lookin at the comments you mentioned: I am rather sure that an account with zero posts which has been created 2 weeks ago is bogus. I am also watching account creation on the bug tracker, and every "valid" account created has been used to ask a question within a day or two.

I don't want to go after account names (there are lots of Suzie1979 where I come from, and those are great gals) or even trying to check the URLs they add to the accounts.

As said: Account has been created, has zero posts and is > two weeks old: That should be a spam account.

I am getting the mails from users we would delete this way anyway, so I'd say: Let's risk that.
Phil Schaffner

Phil Schaffner

2011-09-29 17:28

reporter   ~0013428

There are legitimate users who create an account and do not post for extended periods, but given the seriousness of the problem we may have to accept a few casualties. We can add a notice to the ReadMe First pages to the effect that accounts with no posts after two weeks will be deleted, but is there a way to add such a notice to the subscription process?
AlanBartlett

AlanBartlett

2011-09-30 00:17

reporter   ~0013429

Currently there are 93,365 activated accounts with zero posts that have been created since August 17th 2011. Sampling a random 25 of those accounts, ALL were bogus/spam and were targets for deletion.

How is the cron job coming along?
AlanBartlett

AlanBartlett

2011-10-01 21:18

reporter   ~0013435

Current status (Sat 1st Oct 2001 @ 2116hours UTC) --

Active accounts: 118,556

Active accounts with zero postings: 96,300

Inactive accounts: 574

Total accounts with zero posting: 96,872

Has there been any progress, please? If no, are you now prepared to give access to those who will work at a resolution of this issue?

user430

2011-10-01 22:12

  ~0013436

Having looked a bit further: There was 1 account with 0 postings older than 60 days, after that the number rose steadily.

I have changed the disclaimer you get to read when you try to register to tell that we reserve the right to delete accounts which are older than two weeks and haven't posted in that time.

The cron job is now running nightly (with our backup) and removes all accounts which are older than two weeks (date -d "two weeks ago") and have 0 posts, I am not looking if an account has been activated or not. So the inactive accounts are removed also.

At the moment there are still ~32000 accounts with not postings. Let's see if that number rises or gets lower.

As the uid is of type MEDIUMINT we still have some headroom, the maximum is 8388607.

user430

2011-10-01 22:13

  ~0013437

I am leaving this open for the moment (and will enjoy the late summer for the rest of the weekend).
AlanBartlett

AlanBartlett

2011-10-01 22:21

reporter   ~0013438

Thank you, Ralph.

Might I suggest that this issue is left open for the entire month of October? Now that we (the moderators) know that something has been configured to minimise this childish annoyance, we will keep the situation under review and (if you are agreeable) post the daily totals so that some form of crude statistics are gathered.
AlanBartlett

AlanBartlett

2011-10-03 01:04

reporter   ~0013442

Mon Oct 3 01:04:28 UTC 2011

Active accounts: 55,473

Active accounts with zero postings: 33,206

Inactive accounts: 859

Total accounts with zero posting: 34,067
AlanBartlett

AlanBartlett

2011-10-04 00:05

reporter   ~0013449

Tue Oct 4 00:03:23 UTC 2011

Active accounts: 55,739

Active accounts with zero postings: 33,461

Inactive accounts: 1,868

Total accounts with zero posting: 35,333
AlanBartlett

AlanBartlett

2011-10-05 01:41

reporter   ~0013462

Wed Oct 5 01:38:16 UTC 2011

Active accounts: 56,166

Active accounts with zero postings: 33,865

Inactive accounts: 2,698

Total accounts with zero posting: 36,564
AlanBartlett

AlanBartlett

2011-10-06 00:28

reporter   ~0013474

Thu Oct 6 00:25:22 UTC 2011

Active accounts: 55,922

Active accounts with zero postings: 33,609

Inactive accounts: 3,314

Total accounts with zero posting: 36,925
AlanBartlett

AlanBartlett

2011-10-07 20:32

reporter   ~0013482

Fri Oct 7 20:29:23 UTC 2011

Active accounts: 54,588

Active accounts with zero postings: 32,255

Inactive accounts: 5,122

Total accounts with zero posting: 37,376

user430

2011-10-07 21:04

  ~0013484

So the account numbers roughly stay the same, except for the inactive accounts (as there are none older than two weeks).

That should also stay on a steady level, once accounts in that category get deleted.
AlanBartlett

AlanBartlett

2011-10-07 21:08

reporter   ~0013485

@Range -- Agreed. Please let's continue to monitor and log the stats. (The idea of "carrying around" 30,000+ of "dead wood" isn't my idea of fun.)
AlanBartlett

AlanBartlett

2011-10-08 19:24

reporter   ~0013496

Sat Oct 8 19:22:26 UTC 2011

Active accounts: 54,116

Active accounts with zero postings: 31,777

Inactive accounts: 5,728

Total accounts with zero postings: 37,509
AlanBartlett

AlanBartlett

2011-10-09 19:41

reporter   ~0013502

Sun Oct 9 19:38:40 UTC 2011

Active accounts: 53,985

Active accounts with zero postings: 31,633

Inactive accounts: 6,254

Total accounts with zero postings: 37,888
AlanBartlett

AlanBartlett

2011-10-10 21:30

reporter   ~0013513

Mon Oct 10 21:27:46 UTC 2011

Active accounts: 54,410

Active accounts with zero postings: 32,051

Inactive accounts: 7,376

Total accounts with zero postings: 39,430
AlanBartlett

AlanBartlett

2011-10-11 19:43

reporter   ~0013518

Tue Oct 11 19:40:40 UTC 2011

Active accounts: 54,188

Active accounts with zero postings: 31,815

Inactive accounts: 8,065

Total accounts with zero postings: 39,882
AlanBartlett

AlanBartlett

2011-10-14 01:33

reporter   ~0013538

Oops. Where did the reports for Wednesday and Thursday go? (Rhetorical question.)

Fri Oct 14 01:30:46 UTC 2011

Active accounts: 54,617

Active accounts with zero postings: 32,223

Inactive accounts: 9,550

Total accounts with zero postings: 41,774
AlanBartlett

AlanBartlett

2011-10-15 00:09

reporter   ~0013545

Sat Oct 15 00:03:00 UTC 2011

Active accounts: 54,601

Active accounts with zero postings: 32,199

Inactive accounts: 10,196

Total accounts with zero postings: 42,400
AlanBartlett

AlanBartlett

2011-10-16 01:47

reporter   ~0013546

Sun Oct 16 01:45:05 UTC 2011

Active accounts: 54,728

Active accounts with zero postings: 32,312

Inactive accounts: 10,936

Total accounts with zero postings: 43,249
AlanBartlett

AlanBartlett

2011-10-18 00:27

reporter   ~0013562

Tue Oct 18 00:25:33 UTC 2011

Active accounts: 54,995

Active accounts with zero postings: 32,564

Inactive accounts: 11,686

Total accounts with zero postings: 44,258
AlanBartlett

AlanBartlett

2011-10-20 01:24

reporter   ~0013579

Thu Oct 20 01:19:12 UTC 2011

Active accounts: 55,402

Active accounts with zero postings: 32,957

Inactive accounts: 12,318

Total accounts with zero postings: 45,280
AlanBartlett

AlanBartlett

2011-10-21 20:47

reporter   ~0013594

Fri Oct 21 20:45:19 UTC 2011

Active accounts: 53,340

Active accounts with zero postings: 30,851

Inactive accounts: 11,957

Total accounts with zero postings: 42,810
AlanBartlett

AlanBartlett

2011-10-23 15:33

reporter   ~0013601

Sun Oct 23 15:30:30 UTC 2011

Active accounts: 50,445

Active accounts with zero postings: 27,933

Inactive accounts: 11,871

Total accounts with zero postings: 39,806
AlanBartlett

AlanBartlett

2011-10-24 04:08

reporter   ~0013604

Mon Oct 24 04:06:05 UTC 2011

Active accounts: 48,889

Active accounts with zero postings: 26,376

Inactive accounts: 11,730

Total accounts with zero postings: 38,107
AlanBartlett

AlanBartlett

2011-10-25 00:13

reporter   ~0013617

Tue Oct 25 00:11:18 UTC 2011

Active accounts: 49,538

Active accounts with zero postings: 27,011

Inactive accounts: 12,237

Total accounts with zero postings: 39,254
AlanBartlett

AlanBartlett

2011-10-26 22:11

reporter   ~0013641

Wed Oct 26 22:08:39 UTC 2011

Active accounts: 46,009

Active accounts with zero postings: 23,447

Inactive accounts: 11,746

Total accounts with zero postings: 35,193
AlanBartlett

AlanBartlett

2011-10-27 23:37

reporter   ~0013647

Thu Oct 27 23:34:00 UTC 2011

Active accounts: 44,196

Active accounts with zero postings: 21,615

Inactive accounts: 11,692

Total accounts with zero postings: 33,306
AlanBartlett

AlanBartlett

2011-10-27 23:43

reporter   ~0013648

@Range, Looking back in the notes, I see you set up the cron job using the following algorithm --

[quote]
The cron job is now running nightly (with our backup) and removes all accounts which are older than two weeks (date -d "two weeks ago") and have 0 posts, I am not looking if an account has been activated or not. So the inactive accounts are removed also.
[/quote]

I wonder if it would now be worth "tuning" the process, i.e. reducing the "grace" period?

To me, 33,306 is still a lot of "dead wood".
AlanBartlett

AlanBartlett

2011-10-28 21:10

reporter   ~0013661

Fri Oct 28 21:10:01 UTC 2011

Active accounts: 42,115

Active accounts with zero postings: 19,525

Inactive accounts: 11,640

Total accounts with zero postings: 31,165
toracat

toracat

2011-10-28 23:52

manager   ~0013664

Reminder sent to: user430

Could you respond to Alan's query in note 13648 please?

user430

2011-10-29 09:33

  ~0013667

On the other hand I'd like to give people a certain time to do their "first post". 30000 dead accounts shouldn't really weigh the DB down - most performance problems I've seen have been around removing old sessions - but that means users visiting the site and not dead wood.
AlanBartlett

AlanBartlett

2011-10-29 23:10

reporter   ~0013678

Sat Oct 29 23:10:28 UTC 2011

Active accounts: 40,115

Active accounts with zero postings: 17,514

Inactive accounts: 11,529

Total accounts with zero postings: 29,044
AlanBartlett

AlanBartlett

2011-10-30 19:46

reporter   ~0013683

Sun Oct 30 19:44:06 UTC 2011

Active accounts: 38,357

Active accounts with zero postings: 15,752

Inactive accounts: 11,359

Total accounts with zero postings: 27,114
AlanBartlett

AlanBartlett

2011-10-31 19:26

reporter   ~0013685

Mon Oct 31 19:26:39 UTC 2011

Active accounts: 36,740

Active accounts with zero postings: 14,119

Inactive accounts: 11,508

Total accounts with zero postings: 25,631
AlanBartlett

AlanBartlett

2011-11-01 19:47

reporter   ~0013689

As the month of October 2011 has now ended, I shall stop posting the daily statistics and ask that all interested parties review the current status.

After a suitable period of time to allow for comments, I will then suggest that this issue be closed.

user430

2011-11-02 23:52

  ~0013696

Thanks for all that work. I guess we'll get to around 20.000 bogus accounts at any given time on the system. I personally think this is a number which we can live with.

user430

2013-01-28 21:44

  ~0016357

Okay, I think we all now live with that for more than one year :)

Closing

Issue History

Date Modified Username Field Change
2011-09-14 15:38 Phil Schaffner New Issue
2011-09-14 15:47 toracat Note Added: 0013259
2011-09-14 19:36 CiaW Note Added: 0013261
2011-09-14 21:52 AlanBartlett Note Added: 0013264
2011-09-14 22:12 user430 Note Added: 0013266
2011-09-14 23:43 CiaW Note Added: 0013268
2011-09-15 00:02 kbsingh@karan.org Note Added: 0013269
2011-09-15 00:44 AlanBartlett Note Added: 0013270
2011-09-15 13:25 Phil Schaffner Note Added: 0013275
2011-09-15 14:30 Phil Schaffner Note Added: 0013277
2011-09-17 17:53 kbsingh@karan.org Note Added: 0013294
2011-09-17 19:36 AlanBartlett Note Added: 0013296
2011-09-18 00:44 kbsingh@karan.org Note Added: 0013299
2011-09-18 02:13 AlanBartlett Note Added: 0013301
2011-09-18 03:10 Phil Schaffner Note Added: 0013302
2011-09-18 22:37 kbsingh@karan.org Note Added: 0013305
2011-09-18 23:23 AlanBartlett Note Added: 0013306
2011-09-19 14:47 Phil Schaffner Note Added: 0013313
2011-09-19 16:21 AlanBartlett Note Added: 0013317
2011-09-19 21:14 user430 Note Added: 0013320
2011-09-20 14:27 Phil Schaffner Note Added: 0013326
2011-09-26 19:45 Phil Schaffner Note Added: 0013401
2011-09-26 23:49 AlanBartlett Note Added: 0013403
2011-09-28 03:14 AlanBartlett Note Added: 0013410
2011-09-28 03:53 toracat Note Added: 0013411
2011-09-28 21:45 user430 Note Added: 0013419
2011-09-28 22:59 AlanBartlett Note Added: 0013420
2011-09-28 23:18 user430 Note Added: 0013421
2011-09-29 01:24 toracat Status new => assigned
2011-09-29 17:28 Phil Schaffner Note Added: 0013428
2011-09-30 00:17 AlanBartlett Note Added: 0013429
2011-10-01 21:18 AlanBartlett Note Added: 0013435
2011-10-01 22:12 user430 Note Added: 0013436
2011-10-01 22:13 user430 Note Added: 0013437
2011-10-01 22:21 AlanBartlett Note Added: 0013438
2011-10-03 01:04 AlanBartlett Note Added: 0013442
2011-10-04 00:05 AlanBartlett Note Added: 0013449
2011-10-05 01:41 AlanBartlett Note Added: 0013462
2011-10-06 00:28 AlanBartlett Note Added: 0013474
2011-10-07 20:32 AlanBartlett Note Added: 0013482
2011-10-07 21:04 user430 Note Added: 0013484
2011-10-07 21:08 AlanBartlett Note Added: 0013485
2011-10-08 19:24 AlanBartlett Note Added: 0013496
2011-10-09 19:41 AlanBartlett Note Added: 0013502
2011-10-10 21:30 AlanBartlett Note Added: 0013513
2011-10-11 19:43 AlanBartlett Note Added: 0013518
2011-10-14 01:33 AlanBartlett Note Added: 0013538
2011-10-15 00:09 AlanBartlett Note Added: 0013545
2011-10-16 01:47 AlanBartlett Note Added: 0013546
2011-10-18 00:27 AlanBartlett Note Added: 0013562
2011-10-20 01:24 AlanBartlett Note Added: 0013579
2011-10-21 20:47 AlanBartlett Note Added: 0013594
2011-10-23 15:33 AlanBartlett Note Added: 0013601
2011-10-24 04:08 AlanBartlett Note Added: 0013604
2011-10-25 00:13 AlanBartlett Note Added: 0013617
2011-10-26 22:11 AlanBartlett Note Added: 0013641
2011-10-27 23:37 AlanBartlett Note Added: 0013647
2011-10-27 23:43 AlanBartlett Note Added: 0013648
2011-10-28 21:10 AlanBartlett Note Added: 0013661
2011-10-28 23:52 toracat Note Added: 0013664
2011-10-29 09:33 user430 Note Added: 0013667
2011-10-29 23:10 AlanBartlett Note Added: 0013678
2011-10-30 19:46 AlanBartlett Note Added: 0013683
2011-10-31 19:26 AlanBartlett Note Added: 0013685
2011-11-01 19:47 AlanBartlett Note Added: 0013689
2011-11-02 23:52 user430 Note Added: 0013696
2013-01-28 21:44 user430 Note Added: 0016357
2013-01-28 21:44 user430 Status assigned => resolved
2013-01-28 21:44 user430 Resolution open => fixed