Word Frequency Tool

Status
Not open for further replies.

Tdol

Editor, UsingEnglish.com
Staff member
Joined
Nov 13, 2002
Member Type
Native Language
British English
Home Country
UK
Current Location
Japan
We have added a Word Frequency Analyser in the Members' Area. You simply copy and paste the text you want analysed in the box and click Calculate Now! and the results will be displayed automatically. The first box will display...

Read more...

Source: TDOL's Language Archive
 

happyhel

Member
Joined
Feb 23, 2005
Thats great, but you have to register again

It will not let me in with the same Login ID and password
 

Tdol

Editor, UsingEnglish.com
Staff member
Joined
Nov 13, 2002
Member Type
Native Language
British English
Home Country
UK
Current Location
Japan
We're working on it. Sorry for the inconvenience. ;-)
 

Tdol

Editor, UsingEnglish.com
Staff member
Joined
Nov 13, 2002
Member Type
Native Language
British English
Home Country
UK
Current Location
Japan
Glad you like it. ;-)
 

jjb529

New member
Joined
Nov 27, 2006
Member Type
Academic
I recently wrote a small (free) windows application that is (among other things) a word frequency tool. It is opensource, and the application and source can be downloaded at:

http://textstats.sourceforge.net

I hope someone can find this useful ;-)
 

abasher

New member
Joined
Dec 11, 2006
Member Type
Student or Learner
jjb529, Thank you!

For me this little piece of program is a Gold mine! I'm currently doing a Master Thesis in Text Mining, and you would believe what problems I've had finding free tools to use for document statistics. The one on this website cannot handle the length of document I want to analyse (it says documents up to 500KB, but refuses mine because they are over 10,000 characters).

Three things that I would like to see in the program:
1) The possibility to choose many documents at once, possibly with the ability to see deviances between them.
2) Some sort of stemming available. There are many free ones around (such as http://www.tartarus.org/~martin/PorterStemmer/), so the amount of work should be minimal.
2) The Ignore*.txt files working. They aren't working at the moment, right? If they are, I'm failing to see how to get them activated.

Again, thanks a bunch. Keep up the good work!
 
Last edited by a moderator:

Red5

Webmaster, UsingEnglish.com
Staff member
Joined
Nov 13, 2002
Member Type
Native Language
British English
Home Country
England
Current Location
England
Hi. I've just increased the limit to 50,000 characters for the Members' Area Text Analyser.
 

sauron123

New member
Joined
Feb 7, 2007
Member Type
Student or Learner
i discovered a similar tool that adds a scoring ability to the document being analyzed. It adds a whole new, tangible dimension to the analysis, rather than just blindly counting words, letters etc: getfirefly.net
 

Red5

Webmaster, UsingEnglish.com
Staff member
Joined
Nov 13, 2002
Member Type
Native Language
British English
Home Country
England
Current Location
England
:hi: The download you mention looks quite good, but complicated. Could you explain a little more about the scoring you mention? ;-)
 

SSteffin

Member
Joined
Jan 6, 2007
Member Type
Other
There are tons of free text analysis/data mining tools available. While most are concordance tools, there are some which provide linking features. Here is one site that reviews much of what is available by category: Text Analysis Info.

You may also search for Plagiarism Tools. Some of these are free, such as CopyFind v2.6 which you can directly find with Google.

Also, if you do the search: text analysis software free, there is enough to keep you busy for a long time.
For building a web bibliography of what’s available, enter the same search in the Ultimate Research Assistant (Ultimate Research Assistant (Web Edition)) and you will get a very complete listing of available text tools.


You may also want to remove the FREE from either/both of your searches, giving you a much wider selection of what’s out there. If I can be of further assistance, please let me know, via a private email or message.

As a now retired statistician, I have been heavily involved in developing text analysis tools, and will be happy to share some of my experience with you.
 

jjb529

New member
Joined
Nov 27, 2006
Member Type
Academic
Hmm, I did discover that the ignore words weren't being properly ignored. Thanks for commenting on this. It appears that I was looking for them in the wrong place (on the root of the c drive). They actually existed there on my machine, but of course the installer doesn't put them there. The most recent release should have this fixed (as in the ignore lists are looked for in the location that the .exe is installed too).

I only recently noticed your comment. If you want to make comments so that I can actually see them you should report a bug on the sourceforge website at:

http://sourceforge.net/projects/textstats

This way I will actually get an email that someone has reported a problem.
 
Status
Not open for further replies.
Top