Looking for a method to summarise a sentence.

[jroehl]

Executive Summary:

We are looking for technology that produces phrases that summarize a sentence.

More detail:

This is not a normal technical question. But we have been seeking a technical solution for several months now. And it may have components of artificial intelligence. Which we find both interesting and challenging. We hope it is one of the more interesting questions asked on StackOverflow.com. But thankfully the question is very simple:

For any one English sentence, find phrases of 2 to 5 words that might summarize that sentence.

The phrase should be pithy and draw a user’s attention to it, when presented with other similar phrases. The phrase should be a good cognitive digest of the substance of the sentence and therefore the underlying topic that the sentence is a component of. We would like to develop a computer algorithm to accomplish this task.

We understand this to be a unique requirement. This is because it is a potentially crucial component for the website:

timelinesinhistory.com

We only mention the website because many times responders at StackOverflow.com ask for better examples to make questions less ambiguous. Our intent is not to hawk this website.

This website takes historical textual information (white papers, articles and books), breaks down this information into chapters, paragraphs and sentences. It then finds dates in the sentences and allows the user to place this information on a timeline. At this point the users are prompted to either type in or copy and paste a sentence summary to be placed on the timeline. Because typically placing a complete sentence on the timelines is both visibly unappealing and non-standard.

For now, if you would like a functional use case, you might go to timelinesinhistory.com. Click on any colored plus sign to the left of any article and drill down to a sentence. You may turn on any sentence that has a date in it (it will be red). Any sentence that is green is on the timeline and may be edited by clicking on the little hamburger menu at the beginning of the sentence. Right now, when you turn a sentence “on” we automatically place the first 3 words in the sentence on the timeline. This seldom works for this application and the user is typically forced to copy and paste a phrase from the sentence that is being selected. This is what we are trying to make more efficient for this website’s users.

We have databases with thousands of nouns, adjectives and verbs. We obviously can detect where the date is in the sentence and which words in the sentence are capitalized. We seek a method to deduce 1 to 3 phrases in the sentence that best describes that sentence or draw attention to it. We would like to know:

1) What this subset of linguistics might be called (sentence summarization)?

2) What patterns of sentence diagraming might be used to accomplish our goal (remember that in school)?

3) Has anybody else ever attempted this?

4) Might there be a scholar in the English language that might be interested in this topic?

5) Are there any obscure algorithms we might be able to adopt, buy or utilize to accomplish our goal?

We have tried textrazor.com but it just is not quite what it was built for.

So, here are some examples of “sentence summaries” submitted by users of the website for an article on Michelangelo. There is no right answer here, it is somewhat subjective. And that is what makes it that much more challenging.

A large list of sentence summary examples can be found at:

http://timelinesinhistory.com/sentencesummaries.htm

Here is a couple of examples {summary phrase} = {sentence}:

• {Humanist academy} = {From 1490 to 1492, Michelangelo attended the Humanist academy the Medici had founded along Neo-Platonic lines.}
• {Poet Vittoria Colonna} = {It was at this time that he met the poet Vittoria Colonna, marchioness of Pescara, who was to become one of his closest friends until her death in 1547.}
• {Ceiling of the Sistine Chapel} = {During the same period, Michelangelo painted the ceiling of the Sistine Chapel, which took approximately four years to complete (1508-1512).}
• {Humanist academy} = {From 1490 to 1492, Michelangelo attended the Humanist academy the Medici had founded along Neo-Platonic lines.}
• {David} = {Michelangelo responded by completing his most famous work, the statue of David, in 1504.}

Thank you for your time and consideration

Jeff Roehl

 

[jutfrank]

Hi Jeff

My feeling is that your goal is not possible—at least with any decent degree of reliability.

The only way I can even imagine it being achievable is to somehow extract topic sentences from the text using some kind of semantic analysis. But the problem as I see it is that not only is it often extremely hard to identify topic sentences in the first place, they often don't appear in any given passage at all.

Note that I'm not a linguist and I know very little about machine learning, but I have spent a lot of time summarising and teaching other people to summarise passages of texts. I hope you're not too discouraged by my frank response, and I really hope I'm wrong. Good luck.

 

[jroehl]

Hi Jeff

My feeling is that your goal is not possible—at least with any decent degree of reliability.

The only way I can even imagine it being achievable is to somehow extract topic sentences from the text using some kind of semantic analysis. But the problem as I see it is that not only is it often extremely hard to identify topic sentences in the first place, they often don't appear in any given passage at all.

Note that I'm not a linguist and I know very little about machine learning, but I have spent a lot of time summarising and teaching other people to summarise passages of texts. I hope you're not too discouraged by my frank response, and I really hope I'm wrong. Good luck.

Thank you for replying to my question.

Appently, this very simple concept is much more complex than what it appears to be.

Thanks
Jeff

 

[emsr2d2]

Apparently, this very simple concept is much more complex than [STRIKE]what[/STRIKE] it appears to be.

Thanks.

See above.

 

[Tdol]

I just get a blank page for the timelines site.