Feature Request - cooking up next version
Goto page Previous  1, 2
 
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.   printer-friendly view    phpMiX.org Forum Index -> mxBB Modules -> mxBB Module: mx_errordocs
View previous topic :: View next topic  
Author Message
dcz
Apprentice
Apprentice


Joined: 14 Mar 2006
Posts: 15

PostPosted: Wed Apr 05, 2006 4:16 am    Post subject: Reply with quote

Al right, here is first draft : http://www.marsatak.org/download/mx_errordocs-securedocs_PRE_BETA_0.1.zip

Known bug : log sorting in acp view, does not work, I did not work on it so far.
TODO : Fine tunning of the scoring system, I have to perform wider test for this. So far, as you can see, I added some debug array and output to be able to see where does the score (evnt_tag) increase and why.
Have to work a bit more on more accurate conditions to have it increase too.
Also, no auto ban nor session removal implemented, since this require the scoring system to be fine tuned.
I also need to finish up the report mode (abitlity to ask for report directly).

But, besides that, the main scritp is working and installing great.
The bot trapping mod needs some trap to be set up, my first idea so far is to use something like header("Location: BOTTRAPPED"); in a hooney pot file (forbiden with robots.txt for example). This mainly to find out easy which file was tryed thanks to the uri which leads to the good folder (would not be the case if we'd redirected to the errordoc file, this create a 404 that gets catched up by the mod anyway, but with proper infos.

Try to put "BOTTRAPPED" or "SECURED" (same idea but to replace the phpbb die('hacking attemps'), considered and counted as a hack attemps) in a test url.

As far as installing everything as usual, not tested on phpbb stand alone though, but I'll work on it, the code is already ment ot work for both.
You have to set up the provided rules in the .htaccess and also, I am using mail template and I did not find a way not to put them into phpbb lang folders (is there any you know?).

I had to get rid of the portal page mod, since it would not be as secure to filter uri for the rewritecond (in order to still allow the errodocs page loading while those url are redirected), in standalone mod we filter file name, more acurate.

I changed the db settings a bit, using auto increment and a different primary key (the original tstamp primary key was not allowing more than one log per sec, where auto incremented evnt_id allows super fast update).

The scoring system is in tunning mode, with just one ip, it will be very hard to see more than three pages before the fast exit, I know it is frustrating, comment the update config code to have more time to experiment, and obviously if you whish to.

Anyway, I installed it on the same test server I told you about. You won't see much difference besides the script is more agressive.

I also Added severall thing to the mails sent.
here is an example :
Code:
Possible Security Alert : Event Logged
---------------------------------------------------------

SecureDocs ~ ErrorDocs Alert  on "mxBB-Portal" ( http://localhost/ )

This could be a false alert, but could not too.

There are 9 logged error related to this event so far.
These related logs have an average security scoring of : 107
The related events average time difference is : 52
An average related score above 20 is to be taken under consideration.
An average related event time difference under 5 sec is to be taken under consideration.

This email was sent because one or many conditions matched the event.
Your general security level is set to 1, 3 being the highest.

Remeber, an alert is being sent only in case of :
    - Hacking attemps : Default Security;
       & Bot trapping : High Security;
         & Bad Bots & Banned ip : Max Security.

Or if 5 errors where most likely to be linked to one person or if the event scoring is too high.

Two users with the same ip is to be looked at.
Several ip obviously linked to the same event, itself looking sucpicious as far as requested uri, should lead to a ban.


What follow are logs of the event, you may recieve more than one mail if the scan appear to be massive.
Further analysis of those and the server log if any will help you out taking the right decision.


/*****************************************************************************************************************************/
/**************************************************EVENT_LOG******************************************************************/
/*****************************************************************************************************************************/
||___________________________________________________________________________________________________________________________||
||_______ERROR_TIME______||__ERR___||___UID___||_______IP________||_SCORE_||______PROXY_______||______HOST___________________||
||_____________________________________________________USER_AGENT____________________________________________________________||
||_REQUESTED_URI__________________________________||_HTTP_REFERER____________________________________________________________||
||___________________________________________________________________________________________________________________________||
|| 2006-04-05 @ 01:35:16 || 404    || UID : 2 || 127.0.0.1    || 197    || _NotProXy_    || killa    ||
 || Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1    ||
|| /fhdhdfhdh    || _EMPTY_HTTP_REFERER_    ||

||___________________________________________________________________________________________________________________________||
/*****************************************************************************************************************************/
/*****************************************************************************************************************************/

/*******************************************************/
/********************LAST_TWO_QUICK_LOGS****************/
/*******************************************************/
||_____________________________________________________||
||_______ERROR_TIME______||__SCORE__||_______IP________||
||_____________________________________________________||
|| 2006-04-05 @ 01:32:36 || 1050    || 127.0.0.1    ||
|| _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ||
|| 2006-04-05 @ 01:32:11 || 1050    || 127.0.0.1    ||
||_____________________________________________________||
/*******************************************************/
/*******************************************************/

/*****************************************************************************************************************************/
/*****************************************************RELATED_LOG*************************************************************/
/*****************************************************************************************************************************/
||___________________________________________________________________________________________________________________________||
||_______ERROR_TIME______||__ERR___||___UID___||_______IP________||_SCORE_||______PROXY_______||______HOST___________________||
||_____________________________________________________USER_AGENT____________________________________________________________||
||_REQUESTED_URI__________________________________||_HTTP_REFERER____________________________________________________________||
||___________________________________________________________________________________________________________________________||
|| /fhdhdfhdh    || _EMPTY_HTTP_REFERER_ ||
||___________________________________________________________________________________________________________________________||
|| 2006-04-05 @ 01:21:49 || 404    || UID : 2 || 127.0.0.1    || 196     || _NotProXy_    || _NotCalC_    ||
|| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1    ||
|| /fhdhdfhdh    || _EMPTY_HTTP_REFERER_ ||
||___________________________________________________________________________________________________________________________||
|| 2006-04-05 @ 01:21:21 || 404    || UID : 2 || 127.0.0.1    || 190     || _NotProXy_    || _NotCalC_    ||
|| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1    ||
|| /fhdhdfhdh    || _EMPTY_HTTP_REFERER_ ||
||___________________________________________________________________________________________________________________________||
|| 2006-04-05 @ 01:20:11 || 404    || UID : 2 || 127.0.0.1    || 184     || _NotProXy_    || _NotCalC_    ||
|| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1    ||
|| /fhdhdfhdh    || _EMPTY_HTTP_REFERER_ ||
||___________________________________________________________________________________________________________________________||
|| 2006-04-05 @ 01:17:59 || 404    || UID : 2 || 127.0.0.1    || 178     || _NotProXy_    || _NotCalC_    ||
|| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1    ||
|| /fhdhdfhdh    || _EMPTY_HTTP_REFERER_ ||
||___________________________________________________________________________________________________________________________||
|| 2006-04-05 @ 01:17:43 || 404    || UID : 2 || 127.0.0.1    || 173     || _NotProXy_    || _NotCalC_    ||
|| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1    ||
|| /fhdhdfhdh    || _EMPTY_HTTP_REFERER_ ||
||___________________________________________________________________________________________________________________________||
|| 2006-04-05 @ 01:15:29 || 404    || UID : 2 || 127.0.0.1    || 76     || _NotProXy_    || _NotCalC_    ||
|| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1    ||
|| /fhdhdfhdh    || _EMPTY_HTTP_REFERER_ ||
||___________________________________________________________________________________________________________________________||
|| 2006-04-05 @ 01:15:29 || 404    || UID : 2 || 127.0.0.1    || 76     || _NotProXy_    || _NotCalC_    ||
|| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1    ||
|| /fhdhdfhdh    || _EMPTY_HTTP_REFERER_ ||
||___________________________________________________________________________________________________________________________||
|| 2006-04-05 @ 01:15:29 || 404    || UID : 2 || 127.0.0.1    || 76     || _NotProXy_    || _NotCalC_    ||
|| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1    ||
|| /fhdhdfhdh    || _EMPTY_HTTP_REFERER_ ||
||___________________________________________________________________________________________________________________________||
|| 2006-04-05 @ 01:15:29 || 404    || UID : 2 || 127.0.0.1    || 76     || _NotProXy_    || _NotCalC_    ||
|| Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1    ||
|| /fhdhdfhdh    || _EMPTY_HTTP_REFERER_ ||
||___________________________________________________________________________________________________________________________||

/*****************************************************************************************************************************/
/*****************************************************************************************************************************/


A short one is sent in case au massive scan and before it will constatly exit :
Code:
Critical Alert : Massive scan suspected 
---------------------------------------------------------

Some massive and unfriendly scan seems to be going on "mxBB-Portal" ( http://localhost/ )

The ErrorDocs / SecureDocs Self Defense Engine was put on by this event.

The event scoring so fart is : 70.
Current error performed is : 404.

The script ended up shortly to preserve server ressources while preparing appropriate counter mesures.
Here are the short logs of the event :

/*****************************************************************************************************************************/
/**************************************************EVENT_LOG******************************************************************/
/*****************************************************************************************************************************/
||___________________________________________________________________________________________________________________________||
||_______ERROR_TIME______||__ERR___||___UID___||_______IP________||_SCORE_||______PROXY_______||______HOST___________________||
||_____________________________________________________USER_AGENT____________________________________________________________||
||_REQUESTED_URI__________________________________||_HTTP_REFERER____________________________________________________________||
||___________________________________________________________________________________________________________________________||
|| 2006-04-05 @ 01:40:10 || 404    || UID : 2 || 127.0.0.1    || 70    || _NotProXy_    || _NotCalC_    ||
 || Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.0.1) Gecko/20060111 Firefox/1.5.0.1    ||
|| /fhdhdfhdh    || _EMPTY_HTTP_REFERER_    ||

||___________________________________________________________________________________________________________________________||
/*****************************************************************************************************************************/
/*****************************************************************************************************************************/

/*******************************************************/
/********************LAST_TWO_QUICK_LOGS****************/
/*******************************************************/
||_____________________________________________________||
||_______ERROR_TIME______||__SCORE__||_______IP________||
||_____________________________________________________||
|| 2006-04-05 @ 01:40:10 || 21031241794    || 127.0.0.1    ||
|| _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ||
|| 2006-04-05 @ 01:37:12 || 21031241794    || 127.0.0.1    ||
||_____________________________________________________||
/*******************************************************/
/*******************************************************/

Following attack made under the same ip should not activate any more alert, unless other ip are being used.
The Self Defense Engine will Shorten the process to it's minimum if those IP continue to scan this badly.

If You have activated logs, you should ask for complete analysis for those IP.

// This part needs to be ended  ;-)
To do so : http://localhost/errordocs.php?errlog=fullrep&secretcode=SECRET_CODE&theip=127.0.0.1
Note : You can change this event ip with one of the previous in this url.
Additionnaly you can specify an error to match adding this in the url :
&errno=6668 (6668 stands for Hacking attemps) or any other error number (403, 404 etc ...).

Analyse as well your server's logs to find out more and to take the right decisions as far as updating your banned ip list.

Several ip obviously linked to the same event, itself looking sucpicious as far as requested uri, should lead to a ban.


If you try to install it, you have to set the email to be used before trying it (just pushing submit on time will fill it with the board email,but you can change it to whatever.

Now as far a naming and version number, I don't really know how to do it.
At first I wanted to separate a bit thing between log analysis and output, but in the end I put it all in securedoc_function.php, even though most of your code is still here. And in the end I kept the common.php too, but a lot smaller. I could have put all back in common.php, but I think I'll need more files in the future anyway, so .. Just aknowledge it was not an attemps to say I coded more than I did, just an historical matter.

And I don't know how to call it, I started calling it "ErrorDocs ~ SecureDocs" while the folder remains the same.
Then version number, should I go for 2.0 once done, or start a new dev process (0.1 for example) ?

I let this all for you to decide, also please tell me how you would like to set your copyright throughout the code.


Hope you'll like how it's turning.

++

 
 
_________________
Sig
 
Back to top
View user's profile Send private message
dcz
Apprentice
Apprentice


Joined: 14 Mar 2006
Posts: 15

PostPosted: Mon Apr 17, 2006 12:24 pm    Post subject: Reply with quote

hello markus Wink

Just to tell you I had to delay this dev a bit, but I did fix some bugs, and almost ended the acp part (with sorting options etc ..).

I just need a bit of time to properly tune the scoring system, a bit more complex than I first expected and planned, but I like the way it will be hard to predict for hackers. It should be rather difficult to be able to know how may tries you have before a ban. I hope to have time for this next week.

As soon as I'll get done with the scoring system, I'll issue a beta and start working on security conds and the auto ban system (which will be relying on ua, activated cond (event type) and the event itself (score mostly) ).

But before the auto ban, I want to add backend to list banned ip so that user will be able to regroup banned ip ranges in .htaccess (faster than single ip testing).

Also, please tell me what do I need to do concerning you copyright, where to put it etc ...

I really want it to be clearly defined since I respect your work a lot.

Also, I understand it is hard to follow my script for now (since the scroing sytem of the version I provided is more than agressive (not easy to test), but I'd be very interested in what you think about the path I am currently following.



++

 
 
_________________
Sig
 
Back to top
View user's profile Send private message
Display posts from previous:   
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.   printer-friendly view    phpMiX.org Forum Index -> mxBB Modules -> mxBB Module: mx_errordocs All times are GMT + 1 Hour
Goto page Previous  1, 2
Page 2 of 2


 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum