CeWL

Description

CeWL (pronounce "cool"), the Custom Word List generator is a ruby app which spiders a given url to a specified depth, optionally following external links, and returns a list of words which can then be used for password crackers such as John the Ripper. It has been written by DigiNinja.

Installation

BackTrack

CeWL is already installed in BackTrack and is available in /pentest/passwords/cewl/

From scratch

Gem dependencies

$ sudo gem install http_configuration
$ sudo gem install mime-types
$ sudo gem install mini_exiftool
$ sudo gem install rubyzip
$ sudo gem install spider

Download and install CeWL

$ cd ~/src/
$ wget http://www.digininja.org/files/cewl_4.1.tar.bz2
$ bzip2 -cd cewl_4.1.tar.bz2 | tar xf -

Usage

Syntax

$ cewl [OPTION] ... URL

Options

--help, -h

show help

--keep, -k

keep the downloaded file

--depth <x>, -d <x>

depth to spider to, default 2

--min_word_length <len>, -m <len>

minimum word length, default 3

--offsite, -o

let the spider visit other sites

--write <file>, -w <file>

write the output to the file

--ua <usera-gent>, -u <user-agent>

useragent to send

--no-words, -n

don't output the wordlist

--meta, -a

include meta data

--meta_file <file>

output file for meta data

--email, -e

include email addresses

--email_file <file>

output file for email addresses

--meta-temp-dir <directory>

the temporary directory used by exiftool when parsing files, default /tmp

-v

verbose

Example

Let's say you want to create a password file from http://www.digininja.org/projects/cewl.php and save the password file in passwords.txt. Issue following command

root@bt:/pentest/passwords/cewl# ./cewl.rb -w passwords.txt http://www.digininja.org/projects/cewl.php
checking page http://www.digininja.org/projects/cewl.php/
checking page http://www.FOO.com/FOO/connect/FOONet/Top+Navigator/Home
checking page http://www.digininja.org/styles/styles.css
checking page http://www.digininja.org/rss.xml
[...truncated...]

After a few minutes, you have a passwords file:

root@bt:/pentest/passwords/cewl# head passwords.txt
000
00000
000000000011111111112222222222333
000Z
001
002
007
00Z
00m
0100
root@bt:/pentest/passwords/cewl# wc -l passwords.txt
5118 passwords.txt