During web application testing, it is useful to get the directory and file listing of the root of the web application that you are testing so as to ensure complete coverage of the application.
You can use the below command to get a files and directories listing of the web application root
ls –laR /var/www > cd–filelist.txt
I wrote a simple script to parse and convert the output so that I can pipe the URLs directly to Burpsuite.
The script can be found at https://github.com/milo2012/pentest_scripts/blob/master/web/parseFileList.py
Below is an example of how you can use the script.
python parseFileList.py -f cd-filelist.txt > filelist_out.txt
After running the command, you must modify the filelist_out.txt to search/replace each lines with the FQDN of the website.
E.g. replace /var/www/html/www.domain.com with https://www.domain.com
Next, start Burpsuite and point the proxy listener to 127.0.0.1 port 8080.
The next line will use send each URLs in teh filelist_out.txt to Burpsuite using Curl and Xargs.
cat filelist_out.txt | xargs curl –user-agent “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.152 Safari/537.36” -k -x http://localhost:8080 >/dev/null 2>&1
Sit back and enjoy some coffee as this process could take some time.
I came across GDS Burp API which seems like a very useful tool for parsing Burp Proxy logs. The GDS Burp API exposes a Python object interface to requests/responses recorded by Burp. The below link provides a very good introduction to the API.
I wrote a simple script to use the API to parse the Burp proxy logs and send it to SQLMap to automate testing SQL injection for all GET and POST parameters and skip all urls without any parameters.
1. Clone the GDSSecurity burpee repository git clone https://github.com/GDSSecurity/burpee.git
2. Download burpSQL.py from https://github.com/milo2012/burpSQL into the burpee folder
3. Next, we will have to configure logging in Burpsuite
4. Change the proxy settings of your browser to 127.0.0.1:8080
5. Crawl the website with Owasp Ajax Crawling tool or spider with Burpsuite or the manual way.
Below are the command line options for burpSQL
6. The above is pretty self explanatory. If your Burp proxy log is cluttered with urls from multiple domains, you can filtered the SQL injection testing to specific domains using the –domain switch.
Drop me a message if you have any suggestions or comments. Thank you !