 |
Yes, I urge all CentOS admins who are in charge of operating the web site to look into this problem. |
 |
I'd suggest Capcha and requiring an email confirmation for account creation. I have both on my site (Drupal -based), but also had to block an I.P. for someone who created multiple accounts in an hour or so. I also have much smaller volume. Would something like fail2ban be an option? If it were me, I'd shut down new account creation for a (day, 2 days, even a few hours?) until you can get a handle on it and implement something. Then post a notice on the front page with the reason. Thousands per day is pretty ridiculous. |
|
|
I am also of the opinion that account creation should be disabled until such time as the attack can be thwarted. |
|
|
I spent a complete evening trying to implement the captcha solution we have for the "contact" pages into the user setup pages. To no avail. If anybody has any idea on how to do that with our xoops version - be my guest, I'll happily work with you regarding this problem. |
|
|
In case it's helpful, since I'm not familiar with xoops - I just did a search for 'xoops new user captcha' and found a thread that mentioned Xoops 2.4.1 and Liaise 1.5 issues; and a user pointed to a solution that worked for him (this is the working link): http://jordankasper.com/jquery/captcha/ as well as pointing to another link on that message: http://www.webdesignbeach.com/beachbar/ajax-fancy-captcha-jquery-plugin Later down the page there's a link for a recaptcha module, so here's the original thread: http://www.xoops.org/modules/newbb/viewtopic.php?topic_id=70263&forum=28&post_id=321078 |
 |
how many posts have there been from these accounts ? i.e : how many of these accounts have successfully clicked through, activated their accounts ? if there is a exceptionally low number that answers the above two questions, we might be able to pug in place a script that auto deletes accounts that dont validate within a few hours, or accounts where the initial email does not get delivered. |
|
|
A crude but effective method would be to use a cron job. Currently there are 8174 unactivated accounts. There have been 68922 accounts created since 2011/8/16, all with zero posts. |
|
|
One other wrinkle is that a sizable percentage of the obviously bogus Active accounts show a last login date, while others do not. Apparently at least some of the bots are capable of logging in, so just deleting Inactive accounts, and Active accounts with no last login date, more than X days old would help, but is not a complete solution to getting rid of existing undesirable accounts. |
|
|
Just got done deleting 2117 Inactive accounts more than 7 days old, all created in the last 48 hours since the previous deletion, leaving 6442 less than 7 days old, so the rate on these is currently ~1000/day. The current rate of Active account creation seems to be >2000/day. |
|
|
Can someone quantify this : q: how many of these accounts have successfully clicked through, activated their accounts ? or is there no easy way to work this out ? - KB |
|
|
Obviously we don't want to play "20 questions" but your question itself is ambiguous. "q: how many of these accounts . . ." What do you define as "these accounts", please? At present (Saturday September 17th, 2011 @ circa 1935 hours UTC) this is the current situation: All accounts 96419 Active accounts 88568 Inactive accounts 7848 All suspect/spammer accounts 74361 Active suspect/spammer accounts 66513 Inactive suspect/spammer accounts 7850 (Please note that the figures do not add up precisely due to the fact that new accounts are being adding by the minute.) It is difficult to obtain these figures via the "klunky" control panel interface. The above is the best we can do . . . |
|
|
Alan, since this issue report is about bogus accounts - its a safe bet that we are talking about the bogus accounts. What is the criteria being used to quantify the 'suspect /spammer accounts', the row you have as 74361 ? |
|
|
The current criteria that we are using, following on from a somewhat tedious process of manual examination of a variety of accounts where the "Number of posts is less than one", is an identity constructed of a name+number or number+name (e.g. "peterdotson102" or "1800woodworkin") and the URL field filled with a link to non-CentOS, non-Enterprise Linux or non-computing relevant sites. Such sites are, for example, porn related or offering fake handbags/shoes/clothes or suspect medication or dating services, etc. In terms of battling against this barrage of bot created accounts we were, up until about 6 weeks ago, manually selecting for deletion: (1) "Only active users", "Last login is more than 28 days ago", "Joined date is more than 28 days ago", "Number of Posts is less than 1" and (2) "Only inactive users", "Joined date is more than 7 days ago". That was tedious but it worked well. The logic being that nobody requires an account to access the site, only to make postings. A genuine user would, once the account has been created, log in and, ultimately make her/his posting. We have been working on the assumption that the initial log-in would occur well within a seven day period and the posting would occur within twenty eight days. Due to the way the Control Panel has been configured, we have had to split the seek-find-delete into two separate steps. (An aside -- One of the biggest problems with the Control Panel is that it will only allow 50 records (accounts) to be dealt with at a time.) If there is no immediate solution available, please consider creating two cron jobs, search criteria as (1) and (2) above, to be executed once a day. By implementing such, the inundation can at least be held at a steady state. If execution once a day is not frequent enough, it can become once every 12 hours . . . right down to once an hour. Sorry for my sort-of brain-dump (I am only just getting back into normal mode following on from a health issue) but does the above give sufficient detail with regards to the problem? |
|
|
I fully agree with Alan's statement of the problems, and moderator's approach to dealing with them, except the account names are more general. Many are of the forms: FirstnameLastname - but those never match the email address name. maryXXXX - "mary" plus arbitrary alpha characters. Occasionally other first names. XkindXXrows - LOTS of these, and they reappear after deletion. xxxxxxxxx - semi-arbitrary alpha string, all lower case. XXXXXXXXX - semi-arbitrary alpha string, all capitals. Where X may be an arbitrary letter, sometimes a number. |
|
|
Alan and Phil, thats a good place to start from. Lets also add in : "accounts not activated within 6 hrs of them being created" Let me see if I can come up with a way to harvest this info from the backend db to get some sort of a cron job in place. We should still, imho, try to get a captcha included in the signup process in the longer term. |
|
|
A good idea. Thanks, KB. I guess that brings us back to the subject of "WebSite V2.0" . . . I am a believer that if a fast response is necessary, use the biggest non-destructive hammer first and thereby create some "thinking room". In this case, it would be to disable account creation as a temporary measure. ;) |
|
|
I believe "accounts not activated within 6 hrs of them being created" may be a bit too aggressive, but perhaps our current practice of 7 days is overly conservative. Maybe 3 or 4 days would be reasonable - allowing for creating an account on Friday afternoon and not getting around to activating until the next week. |
|
|
Whereas my belief is that something should be done now, if not a week ago last Wednesday, as long as it is "tunable" / adjustable. Hence my initial suggestion when this was last discussed in the moderator's circle -- "Perhaps it is time for Ralph to disable account creation." :) |
|
|
Sure I can disable account creation (as to accounts not activated in a certain amount of time: I am rather sure that that all happens automatically and that the window is rather smallish). I'd really like to have a captcha solution in there (to at least kick out "dumb" bots), but as said: The tutorials I found were supposed to work with the formcaptcha plugin we already have - but they don't do that on the account creation page, but only on the contact page - and I have no idea as to why that is. Maybe need to revisit this. |
|
|
I'd say definitely - perhaps with a call for help on -devel. Current status: 9545 inactive accounts, 4193 more than 7 days old. 52988 active accounts with zero posts created between 2011/8/16 and 2011/9/13 and no logins at all, or no logins within the last 7 days. |
|
|
Current status: Active Users: 108289 Inactive Users: 13615 8672 inactive accounts more than 7 days old. 65604 active accounts with no posts created more than 7 days ago with no logins in the past 7 days. 86130 total active accounts with no posts. A quick scan of the 16698 active accounts less than 7 days old shows the vast majority are apparently spambot created. |
|
|
Neither Phil nor I are au fait with the intimacies of mysql but, if given access to the raw dbase, we would ensure that deficiency is made good. However, I suspect that access to the dbase could not be provided to us mere mortals! |
|
|
As of the date of this note: 89,170 activated user accounts 14,794 inactivated user account 103,971 total user account fall into the bogus/spamming category. Access via the control panel is now very sluggish. |
|
|
Reminder sent to: Please give a higher priority to this problem. |
|
|
For one, I deleted the inactive accounts from the database, which removed 15000 accounts. I wonder how the spam user accounts can be distinguished - not by mail address, otherwise they wouldn't have been able to activate the accounts. Maybe remove all accounts with zero posts which are older than two weeks? Ralph |
|
|
Ralph, Tedious to do so but please review comments 13301, 13302, 13305, 13306 & 13313. That should then give you an idea of what could/should be used as an algorithm for a deletion cron job(s). If it is possible to disable the ability for *new* account creators to make an account with a link to an external website, that should remove one possible interest to the spambot executors. (Perhaps the ability to insert a website link for existing accounts should be restricted to moderators and above? And while we're at it, ditto for signature blocks?) If it is possible to adjust the control panel logic such that we can perform operations on more than 50 accounts at a time, it would be a help to us moderators. Finally for now, as it seems that the initial assignment of this issue to "donavan" was deemed to be inappropriate, perhaps "range" should now have the pleasure? ;) |
|
|
Yeah, I am sorry, I am still not completely up to par after moving. We cannot restrict url or sig block (but maybe we can take those away from the user creation form) - but I don't think that would stop the automated attacks. Without lookin at the comments you mentioned: I am rather sure that an account with zero posts which has been created 2 weeks ago is bogus. I am also watching account creation on the bug tracker, and every "valid" account created has been used to ask a question within a day or two. I don't want to go after account names (there are lots of Suzie1979 where I come from, and those are great gals) or even trying to check the URLs they add to the accounts. As said: Account has been created, has zero posts and is > two weeks old: That should be a spam account. I am getting the mails from users we would delete this way anyway, so I'd say: Let's risk that. |
|
|
There are legitimate users who create an account and do not post for extended periods, but given the seriousness of the problem we may have to accept a few casualties. We can add a notice to the ReadMe First pages to the effect that accounts with no posts after two weeks will be deleted, but is there a way to add such a notice to the subscription process? |
|
|
Currently there are 93,365 activated accounts with zero posts that have been created since August 17th 2011. Sampling a random 25 of those accounts, ALL were bogus/spam and were targets for deletion. How is the cron job coming along? |
|
|
Current status (Sat 1st Oct 2001 @ 2116hours UTC) -- Active accounts: 118,556 Active accounts with zero postings: 96,300 Inactive accounts: 574 Total accounts with zero posting: 96,872 Has there been any progress, please? If no, are you now prepared to give access to those who will work at a resolution of this issue? |
|
|
Having looked a bit further: There was 1 account with 0 postings older than 60 days, after that the number rose steadily. I have changed the disclaimer you get to read when you try to register to tell that we reserve the right to delete accounts which are older than two weeks and haven't posted in that time. The cron job is now running nightly (with our backup) and removes all accounts which are older than two weeks (date -d "two weeks ago") and have 0 posts, I am not looking if an account has been activated or not. So the inactive accounts are removed also. At the moment there are still ~32000 accounts with not postings. Let's see if that number rises or gets lower. As the uid is of type MEDIUMINT we still have some headroom, the maximum is 8388607. |
|
|
I am leaving this open for the moment (and will enjoy the late summer for the rest of the weekend). |
|
|
Thank you, Ralph. Might I suggest that this issue is left open for the entire month of October? Now that we (the moderators) know that something has been configured to minimise this childish annoyance, we will keep the situation under review and (if you are agreeable) post the daily totals so that some form of crude statistics are gathered. |
|
|
Mon Oct 3 01:04:28 UTC 2011 Active accounts: 55,473 Active accounts with zero postings: 33,206 Inactive accounts: 859 Total accounts with zero posting: 34,067 |
|
|
Tue Oct 4 00:03:23 UTC 2011 Active accounts: 55,739 Active accounts with zero postings: 33,461 Inactive accounts: 1,868 Total accounts with zero posting: 35,333 |
|
|
Wed Oct 5 01:38:16 UTC 2011 Active accounts: 56,166 Active accounts with zero postings: 33,865 Inactive accounts: 2,698 Total accounts with zero posting: 36,564 |
|
|
Thu Oct 6 00:25:22 UTC 2011 Active accounts: 55,922 Active accounts with zero postings: 33,609 Inactive accounts: 3,314 Total accounts with zero posting: 36,925 |
|
|
Fri Oct 7 20:29:23 UTC 2011 Active accounts: 54,588 Active accounts with zero postings: 32,255 Inactive accounts: 5,122 Total accounts with zero posting: 37,376 |
|
|
So the account numbers roughly stay the same, except for the inactive accounts (as there are none older than two weeks). That should also stay on a steady level, once accounts in that category get deleted. |
|
|
@Range -- Agreed. Please let's continue to monitor and log the stats. (The idea of "carrying around" 30,000+ of "dead wood" isn't my idea of fun.) |
|
|
Sat Oct 8 19:22:26 UTC 2011 Active accounts: 54,116 Active accounts with zero postings: 31,777 Inactive accounts: 5,728 Total accounts with zero postings: 37,509 |
|
|
Sun Oct 9 19:38:40 UTC 2011 Active accounts: 53,985 Active accounts with zero postings: 31,633 Inactive accounts: 6,254 Total accounts with zero postings: 37,888 |
|
|
Mon Oct 10 21:27:46 UTC 2011 Active accounts: 54,410 Active accounts with zero postings: 32,051 Inactive accounts: 7,376 Total accounts with zero postings: 39,430 |
|
|
Tue Oct 11 19:40:40 UTC 2011 Active accounts: 54,188 Active accounts with zero postings: 31,815 Inactive accounts: 8,065 Total accounts with zero postings: 39,882 |
|
|
Oops. Where did the reports for Wednesday and Thursday go? (Rhetorical question.) Fri Oct 14 01:30:46 UTC 2011 Active accounts: 54,617 Active accounts with zero postings: 32,223 Inactive accounts: 9,550 Total accounts with zero postings: 41,774 |
|
|
Sat Oct 15 00:03:00 UTC 2011 Active accounts: 54,601 Active accounts with zero postings: 32,199 Inactive accounts: 10,196 Total accounts with zero postings: 42,400 |
|
|
Sun Oct 16 01:45:05 UTC 2011 Active accounts: 54,728 Active accounts with zero postings: 32,312 Inactive accounts: 10,936 Total accounts with zero postings: 43,249 |
|
|
Tue Oct 18 00:25:33 UTC 2011 Active accounts: 54,995 Active accounts with zero postings: 32,564 Inactive accounts: 11,686 Total accounts with zero postings: 44,258 |
|
|
Thu Oct 20 01:19:12 UTC 2011 Active accounts: 55,402 Active accounts with zero postings: 32,957 Inactive accounts: 12,318 Total accounts with zero postings: 45,280 |
|
|
Fri Oct 21 20:45:19 UTC 2011 Active accounts: 53,340 Active accounts with zero postings: 30,851 Inactive accounts: 11,957 Total accounts with zero postings: 42,810 |
|
|
Sun Oct 23 15:30:30 UTC 2011 Active accounts: 50,445 Active accounts with zero postings: 27,933 Inactive accounts: 11,871 Total accounts with zero postings: 39,806 |
|
|
Mon Oct 24 04:06:05 UTC 2011 Active accounts: 48,889 Active accounts with zero postings: 26,376 Inactive accounts: 11,730 Total accounts with zero postings: 38,107 |
|
|
Tue Oct 25 00:11:18 UTC 2011 Active accounts: 49,538 Active accounts with zero postings: 27,011 Inactive accounts: 12,237 Total accounts with zero postings: 39,254 |
|
|
Wed Oct 26 22:08:39 UTC 2011 Active accounts: 46,009 Active accounts with zero postings: 23,447 Inactive accounts: 11,746 Total accounts with zero postings: 35,193 |
|
|
Thu Oct 27 23:34:00 UTC 2011 Active accounts: 44,196 Active accounts with zero postings: 21,615 Inactive accounts: 11,692 Total accounts with zero postings: 33,306 |
|
|
@Range, Looking back in the notes, I see you set up the cron job using the following algorithm -- [quote] The cron job is now running nightly (with our backup) and removes all accounts which are older than two weeks (date -d "two weeks ago") and have 0 posts, I am not looking if an account has been activated or not. So the inactive accounts are removed also. [/quote] I wonder if it would now be worth "tuning" the process, i.e. reducing the "grace" period? To me, 33,306 is still a lot of "dead wood". |
|
|
Fri Oct 28 21:10:01 UTC 2011 Active accounts: 42,115 Active accounts with zero postings: 19,525 Inactive accounts: 11,640 Total accounts with zero postings: 31,165 |
|
|
Reminder sent to: Could you respond to Alan's query in note 13648 please? |
|
|
On the other hand I'd like to give people a certain time to do their "first post". 30000 dead accounts shouldn't really weigh the DB down - most performance problems I've seen have been around removing old sessions - but that means users visiting the site and not dead wood. |
|
|
Sat Oct 29 23:10:28 UTC 2011 Active accounts: 40,115 Active accounts with zero postings: 17,514 Inactive accounts: 11,529 Total accounts with zero postings: 29,044 |
|
|
Sun Oct 30 19:44:06 UTC 2011 Active accounts: 38,357 Active accounts with zero postings: 15,752 Inactive accounts: 11,359 Total accounts with zero postings: 27,114 |
|
|
Mon Oct 31 19:26:39 UTC 2011 Active accounts: 36,740 Active accounts with zero postings: 14,119 Inactive accounts: 11,508 Total accounts with zero postings: 25,631 |
|
|
As the month of October 2011 has now ended, I shall stop posting the daily statistics and ask that all interested parties review the current status. After a suitable period of time to allow for comments, I will then suggest that this issue be closed. |
|
|
Thanks for all that work. I guess we'll get to around 20.000 bogus accounts at any given time on the system. I personally think this is a number which we can live with. |
|
|
Okay, I think we all now live with that for more than one year :) Closing |
- Anonymous
