Skip to content

Commit 3e452e6

Browse files
Dennis de SwartDennis de Swart
Dennis de Swart
authored and
Dennis de Swart
committed
Initial commit
0 parents  commit 3e452e6

22 files changed

+265493
-0
lines changed

‎README.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
2+
# PHP Moby NLP
3+
4+
Gets the "complete thought" from a verb and a noun using Moby Thesaurus
5+
6+
7+
## What does it do?
8+
- It uses the Moby Thesaurus to get the "complete thought" from a verb and a noun.
9+
- This is meant for Natural Language Processing (NLP) tasks.
10+
11+
How does it work?
12+
- The code looks for "complete thoughts", see the link to Wikipedia here:
13+
```
14+
https://simple.wikipedia.org/wiki/Simple_sentence
15+
```
16+
17+
- First: open the index.php, a form will appear.
18+
- Second: enter a verb, like "walk", "talk" or "run". It is best to user the base form or lemma. So do not use "running", but use "run". This program can be used to get the lemma for you:
19+
```
20+
https://github.com/DennisDeSwart/php-stanford-corenlp-adapter
21+
https://www.phpclasses.org/package/10056-PHP-Natural-language-processing-using-Stanford-server.html
22+
```
23+
24+
- Third, enter a noun like "dog", "cat", "house" or "car".
25+
- Finally, press Submit, wait for the result. The result may take up to 2 seconds.
26+
27+
28+
## What can I do with the result?
29+
30+
- First, you can use it in spam killers: if there are related words in a text, the text is probably not fake.
31+
- Second: you can use this for creating reports and understanding text.
32+
- Third: this could be used as suggestions in search engines.
33+
34+
35+
## Requirements
36+
- PHP 5.3 or higher: it also works on PHP 7
37+
38+
39+
## Installation using Composer
40+
41+
You can install the adapter by putting the following line into your composer.json and running a composer update
42+
43+
```
44+
{
45+
"require": {
46+
"dennis-de-swart/php-moby-nlp": "*"
47+
}
48+
}
49+
```
50+
51+
52+
## Recommended practices
53+
54+
- Looking up words in the thesaurus costs a lot of time, sometimes up to 2 seconds. You should only lookup words if you need to.
55+
- To select the most important words like verbs and nouns, you can use a NLP parser like Stanford's CoreNLP
56+
- To use Stanford CoreNLP check these links:
57+
```
58+
https://github.com/DennisDeSwart/php-stanford-corenlp-adapter
59+
http://stanfordnlp.github.io/CoreNLP/corenlp-server.html
60+
```
61+
62+
63+
## Can I use a different set of words (=thesaurus)?
64+
Yes. This script uses the Moby Thesaurus that is based on the English language from 1996. It doesn't have many modern words. You can change the Thesaurus by using an API.
65+
However, this is not programmed yet. To make it happen, there needs to be an API connection. Also the existing Synonym function needs to be re-written to use this API.
66+
I want to write a script to use the Urban Dictionary API. This would make it talk "urban". Here is an example on how to connect:
67+
```
68+
https://github.com/zdict/zdict/wiki/Urban-dictionary-API-documentation
69+
```
70+
71+
## Example output
72+
73+
See "example_write_document.PNG"
74+
75+
76+
## Any questions?
77+
78+
Please let me know.
79+
80+
81+
## Credits
82+
83+
Brent Rossen, orginal author of the MobyThesaurus.php class
84+
```
85+
https://github.com/phyous/moby-thesaurus
86+
```
87+

‎bootstrap.php‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
<?php
2+
3+
/**
4+
* Log errors: Development purposes only.
5+
*/
6+
error_reporting(E_ALL);
7+
ini_set('display_errors', 1);
8+
9+
/**
10+
* Start composer autoloader
11+
*/
12+
require __DIR__.'/vendor/autoload.php';

‎composer.json‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
"name": "dennis-de-swart/php-moby-nlp",
3+
"type": "library",
4+
"description": "Gets the complete thought from a verb and a noun using Moby Thesaurus",
5+
"keywords": ["Moby","thesaurus","nlp", "natural language", "complete thought"],
6+
"homepage": "https://github.com/DennisDeSwart/PHP-Moby-NLP",
7+
"license": "MIT",
8+
"authors": [
9+
{
10+
"name": "Dennis de Swart",
11+
"email": "dennis@dennisdeswart.nl",
12+
"homepage": "http://www.dennisdeswart.nl",
13+
"role": "Developer"
14+
}
15+
],
16+
"require": {
17+
"php": ">=5.3"
18+
},
19+
"autoload": {
20+
"classmap": ["src/"]
21+
}
22+
}

‎composer.lock‎

Lines changed: 20 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎index.php‎

Lines changed: 220 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,220 @@
1+
<?php
2+
3+
/**
4+
* Instantiate
5+
*/
6+
require_once __DIR__.'/bootstrap.php';
7+
?>
8+
9+
10+
<!-- HEADER -->
11+
<!DOCTYPE html>
12+
<html lang="en">
13+
<head>
14+
<title>PHP Moby NLP</title>
15+
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
16+
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script>
17+
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js"></script>
18+
</head>
19+
<!-- END HEADER -->
20+
21+
22+
<script>
23+
$(document).ready(function() {
24+
$("#btnHelp").click(function(){
25+
document.getElementById("thoughtForm").reset();
26+
$( "#thoughtForm" ).submit(); // submitting an empty form will show the help
27+
});
28+
});
29+
</script>
30+
31+
32+
<?php
33+
if ($_SERVER["REQUEST_METHOD"] == "POST"
34+
&& array_key_exists("verb", $_POST) && !empty($_POST["verb"])
35+
&& array_key_exists("noun", $_POST) && !empty($_POST["noun"])) {
36+
37+
/**
38+
* Get the verb and the noun, do some basic validation
39+
*/
40+
$verb = filter_var($_POST["verb"], FILTER_SANITIZE_STRING);
41+
$noun = filter_var($_POST["noun"], FILTER_SANITIZE_STRING);
42+
43+
/**
44+
* Get the synonym list for the verb and the noun
45+
*/
46+
$verbList = MobyThesaurus::GetSynonyms($verb);
47+
$nounList = MobyThesaurus::GetSynonyms($noun);
48+
49+
/**
50+
* If it can't find the word, set an error message
51+
*/
52+
if(array_key_exists('0', $verbList) && $verbList[0] !== $verb){
53+
// can't find the verb
54+
$errorMessage = 'Can not find this verb. Try another verb';
55+
56+
} elseif(array_key_exists('0', $nounList) && $nounList[0] !== $noun){
57+
// can't find the noun
58+
$errorMessage = 'Can not find this noun. Try another noun';
59+
60+
} else {
61+
62+
// Get complete thoughts
63+
foreach($verbList as $verbValue){
64+
if (in_array($verbValue, $nounList) && $verbValue !== $verb) {
65+
$thoughtMatches[] = array('Thoughts found' => $verbValue);
66+
}
67+
}
68+
69+
// get thoughts text
70+
foreach($verbList as $verbValue){
71+
foreach($nounList as $nounValue){
72+
$regex = '/\b'.$verbValue.'\b/';
73+
preg_match($regex, $nounValue, $match, PREG_OFFSET_CAPTURE);
74+
75+
if($match && $match[0][1] !==0 && $verbValue !== $noun){
76+
$fullThoughtText[] = array('Matched text' => $nounValue, 'Based on thought' => $verbValue);
77+
} elseif (strpos($nounValue, $verbValue) && ($verbValue !== $noun)) {
78+
$partThoughtText[] = array('Partial matched text<br />( this matches only part of the thought,<br /> it is not always correct)' => $nounValue, 'Based on thought' => $verbValue);
79+
}
80+
}
81+
}
82+
}
83+
} elseif (empty($_POST["verb"]) && empty($_POST["noun"])){
84+
// do nothing: help will show
85+
} elseif (empty($_POST["verb"])){
86+
$errorMessage = 'Verb field is empty. Please fill in the verb field.';
87+
} elseif (empty($_POST["noun"])){
88+
$errorMessage = 'Noun field is empty. Please fill in the noun field.';
89+
}
90+
?>
91+
92+
93+
<body>
94+
<div class="container">
95+
96+
<!-- START FORM -->
97+
<h2>PHP Moby NLP: complete thoughts form</h2>
98+
<br>
99+
100+
<form id="thoughtForm" class="form-horizontal" method="post" action=''>
101+
102+
<div class="form-group">
103+
<label class="col-sm-2 control-label">Enter a verb:</label>
104+
<div class="col-sm-6">
105+
<input class="form-control" id="inputVerb" name="verb" type="text" value="">
106+
</div>
107+
</div>
108+
109+
<div class="form-group">
110+
<label class="col-sm-2 control-label">Enter a noun:</label>
111+
<div class="col-sm-6">
112+
<input class="form-control" id="inputNoun" name="noun" type="text" value="">
113+
</div>
114+
</div>
115+
<br />
116+
117+
<input type="submit" value="Submit" class="btn btn-primary btn-lg">
118+
<button style="margin-left: 20px" id="btnHelp" type="button" class="btn btn-info btn-lg">Show help</button>
119+
120+
<br /><br />
121+
</form>
122+
<!-- END FORM -->
123+
124+
125+
<?php
126+
if(!empty($errorMessage)){
127+
128+
echo '<div class="alert alert-danger"><strong>'.$errorMessage.'</strong></div>';
129+
130+
} elseif ($_SERVER["REQUEST_METHOD"] != "POST" || empty($verb) || empty($noun)) {
131+
?>
132+
133+
<!-- START HELP -->
134+
<h3>Examples:</h3>
135+
<ul class="list-group">
136+
<li class="list-group-item">Try <b>verb ="work" </b> and <b>noun = "job"</b>. The result will be thoughts that have to do with working on a job, like "achievement", "assignment" and "labor"</li>
137+
<li class="list-group-item">Try <b>verb ="write"</b> and <b>noun = "document</b>". It will find thoughts like "copy", "draft" and "file" and "print"</li>
138+
</ul>
139+
140+
<h3>How does it work?</h3>
141+
<ul class="list-group">
142+
<li class="list-group-item">The code looks for "complete thoughts", see the link to Wikipedia here:<br />
143+
<a href="https://simple.wikipedia.org/wiki/Simple_sentence">We need at least one verb and one noun together for a complete thought</a></li>
144+
<li class="list-group-item">First enter a verb, like "walk", "talk", "run". It is best to user the base form or lemma. So do not use "running", but use "run". This program can be used to get the lemma for you:
145+
<a href="https://github.com/DennisDeSwart/php-stanford-corenlp-adapter">PHP-Stanford-CoreNLP-Adapter (at GitHub)</a><br />
146+
<a href="https://www.phpclasses.org/package/10056-PHP-Natural-language-processing-using-Stanford-server.html">PHP-Stanford-CoreNLP-Adapter (at PHPClasses.org)</a></li>
147+
<li class="list-group-item">Second, enter a noun like "dog", "cat", "house" or "car".</li>
148+
<li class="list-group-item">Finally, press Submit, wait for the result. The result may take up to 2 seconds.</li>
149+
</ul>
150+
<h3>What can I do with the result?</h3>
151+
<ul class="list-group">
152+
<li class="list-group-item">First, you can use it in spam killers: if there are related words in a text, the text is probably not fake.</li>
153+
<li class="list-group-item">Second: you can use this for creating reports and understanding text.</li>
154+
<li class="list-group-item">Third: this could be used as suggestions in search engines.</li>
155+
</ul>
156+
<h3>I would like to use a different set of words (=thesaurus). Can I do that?</h3>
157+
<ul class="list-group">
158+
<li class="list-group-item">Yes. This script uses the Moby Thesaurus that is based on the English language from 1996. It doesn't have many modern words. You can change the Thesaurus by using an API.
159+
However, this is not programmed yet. To make it happen, there needs to be an API connection. Also the existing Synonym function needs to be re-written to use this API.</li>
160+
<li class="list-group-item">I want to write a script to use the Urban Dictionary API. This would make it talk "urban". Here is an example on how to connect:<br />
161+
<a href="https://github.com/zdict/zdict/wiki/Urban-dictionary-API-documentation">Example of the Urban Dictionary API on Github</a></li>
162+
<li class="list-group-item">Or you could use any other "Thesaurus" API</li>
163+
</ul>
164+
<h3>Please be patient when submitting: the script can take a few seconds</h3>
165+
<ul class="list-group">
166+
<li class="list-group-item">The code needs to search the whole Moby Thesaurus, this can take a few seconds.
167+
This class will probably be a lot faster by using SQL instead of files. However, there is no SQL file of the thesaurus available yet.</li>
168+
</ul>
169+
<!-- END HELP -->
170+
171+
172+
<?php
173+
} else {
174+
?>
175+
176+
<table class="table table-bordered">
177+
<th colspan="3"><h3>
178+
Verb = "<?php echo $verb ?>",
179+
noun = "<?php echo $noun ?>"</h3></th>
180+
<tr>
181+
<td>
182+
<?php
183+
$template = new Template();
184+
if(!empty($thoughtMatches)){
185+
$template->getTable($thoughtMatches, '');
186+
} else {
187+
$template->getTable(array(), '', 'No thought matches found');
188+
}
189+
?>
190+
</td>
191+
<td>
192+
<?php
193+
$template = new Template();
194+
if(!empty($fullThoughtText)){
195+
$template->getTable($fullThoughtText, '');
196+
} else {
197+
$template->getTable(array(), '', 'No thought text was found');
198+
}
199+
?>
200+
</td>
201+
<td>
202+
<?php
203+
$template = new Template();
204+
if(!empty($partThoughtText)){
205+
$template->getTable($partThoughtText, '');
206+
} else {
207+
$template->getTable(array(), '', 'No partial thought text found');
208+
}
209+
?>
210+
</td>
211+
</tr>
212+
</table>
213+
214+
<?php
215+
}
216+
?>
217+
218+
</div>
219+
</body>
220+
</html>

‎moby_example_write_document.PNG‎

37.9 KB
Loading

‎moby_help_screen.PNG‎

78.2 KB
Loading

0 commit comments

Comments
 (0)