Re: Google Ngram Viewer
Thanks birdeen's call,
My style of writing is both brutal and nearly-incorrect due to the simple fact that I have never learned English, I have had only self-approaches, that is I learn it on the fly.
It is useful for me when my mistakes are shown, thanks.
Yes, it is true, I will do some efforts to make the package usable from the desktop and not to force the users to go in prompt by themselves.
Openness is one of my strong qualities, but I intend to use, as always, my own resources. Currently the package is not worth to be uploaded, it is 1day old - I started it yesterday.
That is right, I did feel the gap, it is to be explained for sure, but I wanted to give some overview - it is hard to me to explain the idea(moreover the specifications) while the need is unclear/not-explained. I firmly believe that the ability to make proper word arrangements is the hardcore of English, especially for guys like me who don't want to read study-books but prefer learning by reading texts on daily basis.
Re: Google Ngram Viewer
Your English is very good!
Originally Posted by Sanmayce
Re: Google Ngram Viewer
Enter the 'Graphein' package revision 1
Mini-guide in HTML format, 18 screenshots: here
Mini-guide in PDF format, 4MB: here
The package itself in ZIP format, 757MB: here
Two regimes/modes of operation are available:
- Fully-automatic mode: by running 'Graphein_TXT.bat' you can compare your text files (with .TXT extension placed in _TXT-TREE folder) versus US English Google books 4grams from 2009-07-15;
- Semi(keyboard input is needed)-automatic mode: by running 'GRAPHEIN_keyboard.bat' you can search for your patterns(4grams) into US English Google books 4grams from 2009-07-15.
- Desktop launching;
- In this initial revision the console tools can be executed either from command prompt or desktop via 2 icons;
- In fully-automatic mode files with more than a few thousand words are processed awfully slow;
- Unfortunately US English Google books 4grams from 2009-07-15 made me lose my momentum, not as rich corpus (still) as I expected;
- This revision gives the outline(it is mainly illustrative), it has no future unless some serious approach(mixing Graffith & Leprechaun is a nifty one) is applied;
- It took me 3 days to realize the [f]utility of current (100 brute-force way of comparing), still processing chunks is THE one/main way to go because of 32bit address limitations - I dream of 5x100+ million phrases inserted in million b-trees which demands 64bit code, sadly I am not ready to walk this way, yet.
If anyone finds it useful please feel free to make mirrors at once, I cannot guarantee hosting the .ZIP file for more than month or so.
Re: Google Ngram Viewer
Maybe the easiest way an idea to be understood is to tell its ultimate goal, in a 'I have a dream ...' style.
In my case: I dream of instant English-phrase sidekick (suggester/hinter) tool.
The goal is everybody to have the ability to write, in real time, as correctly as possible.
Nowadays when a text is typed in search-engine fields some pale text/suggestions appear, that is what I am talking about.
Imagine this: A kid playing with words and wanting to construct a sentence by using two afore-chosen words but not knowing any grammar.
The situation is similar to forcing some unexperienced person to drive bicycle instead of tricycle first.
The point is, that even when one is extensively trained, it is never enough - the tricycle remains as a reminder-of-everlasting-ignorance forever.
Or as one of our renowned translators of Jack London has said: 'The difference between the better translator and the good one is in using a dictionary.' - contrary to the expected the former uses it whereas the latter is too 'versed'/(vanity sick) and needs it not, well-said.
Of course for a corporation or skillful programmers it is not a big deal to achieve such functionality.
The problem lies not so much in applied algorithms or used programming language but in the scarcity of data which feeds the suggester.
Currently only Google (I am not their fan) shared their ngram datasets, as far as I know made from 4% of all printed books or 5+ million books.
In my view these huge numbers weigh little because the practice shows one poor (yet) corpus far from needed, to talk here for comprehensiveness is a nonsense (look at 210 unfamiliar 4grams encountered in 'The Little Match Girl').
So my dream needs a NEARLY-UNABRIDGED corpus of English(British, American, Australian ...) 4grams (as minimum) preferably 9grams (enabling whole sentences to appear under your fingers).
I know I know that it looks/sounds like/as a long shot, but such greediness fits well here, after all I speak of billions of phrases, no room for profanity, floppy thinking, half-done encoding.
In my vocabulary 'greedy' and 'uncompromising' are synonyms and I use them interchangeably.
In next paragraphs I tried to visualize poorly one static suggestion for word 'getting' by giving all/PROPER three-words-collocations from left and right side:
to get the suggestions I ran Graphein in search for:
and finally the first search yields 17,425 hits, while the second search yields 18,773 hits.
The results for '*_getting|':
The results for 'getting_*':
Having all these suggestions is not enough, here comes the last but not least part: visualization.
Well, I cannot show anything but the suggestive outcome is something like:
Step 1: write 'getting'
two pop-up flowing windows appear each filled with above suggestions.
Step 2: you write 'in getting'
the useful part here is that the entered 2gram is to be searched not into 4grams but say 5grams (3 words from both sides, again) and so on until a suitable sentence (or 9gram) appears.
For example one suggestion is: 'helpful in getting information'.
Step 3: you have obtained a stable/proper 4gram containing needed words from left and right, and you finish the sentence by yourself.
The magical part is that (even if you are not sure what you are looking for) the sheer abundance of phrases will guide/supply/form your thought dynamically i.e. writing and thinking will happen (hopefully) simultaneously.
I recall a funny scene from 'Captain Apache' movie in which Lee Van Cleef was being asked 'What are you searching?' and his answer was something like: 'Nothing. It is amazing how many things pop-up when you just search.'
Let there be greediness ...
The style choosing would be try-once-give-me-more i.e. addictive/indispensable, as in the next example:
getting_a_handle_on [Google books]
getting_a_handle_on [Australian magazines & newspapers]
getting_a_handle_on [Agatha Christie - anthology]
getting_a_handle_on [Arthur Conan Doyle - 'Sherlock Holmes' collection]
getting_a_handle_on [Irish legends]
getting_a_handle_on [Fairy tales translated into English]
Obviously relying on one corpus alone is not remotely as useful as using hundreds-why-not-thousands of corpora.
It must feature smooth scrolling and animated-like appearance, but this is up to Graphic-Interface designers and gadget manufacturers.
Yes, the futuristic way of writing is neither left-to-right nor right-to-left anymore but from middle-to-edges, he-he.
I believe/KNOW the time (for such an assistant) is nearer than most of us think.
Search Engine Optimization by vBSEO