Home

Welcome to the ANN4j wiki!

This package provides Object oriented Neural Networks for making Explainable Networks. Object Oriented Network structure is helpful for observing each and every element the model. This package is developed for XAI research and development.

Usage

Download

Releases

The package can be imported after download. import ann4j.*;

Requirements and Dependencies

None. This package is made using 100% Pure Java. The java package requires java 5.0+. No other requirements are required. Recommended to use the latest version of Java. Java download link

Training

Setting parameters

Setting the output file to be output.txt and enabling command line logging.
```
	parameter.setOutputFile("output.txt", true);
```

Setting the number of neurons in each layer.

	parameter.setLayerArray(784, 32, 16, 16, 26);

Setting the training file to be emnist-letters-train.csv and the file type
```
	parameter.setTrainingFileReader("emnist-letters-train.csv", "mnist");
```

Setting the testing file

	parameter.setTestingFileReader("emnist-letters-test.csv", "mnist");

Setting the learning rate for weights
```
	parameter.setLearningRate(1);
```
Setting the learning rate for the bias to 1.
```
	parameter.setBiasLearningRate(1);
```
Setting the epsillion value for the relevance propagation algorithm.
```
	parameter.setEpsillion(0);
```
Setting the batch size
```
	parameter.setBatchsize(10);
```

Setting the rectification function.

	parameter.setRectificationFunction("sigmoid");

Training the Model

Creating a new instance of the Trainer class.
```
	Trainer myTrainer = new Trainer();
```
Training the network with 88800 samples for n epochs
```
	myTrainer.train(m, n);
```
Creating a new instance of the NeuronObserver class this class will observe the neurons and respond when every parameter is changed.
```
	NeuronObserver myNeuronObserver = new NeuronObserver();
```
Testing the network with 9990 samples.
```
	myTrainer.test(9990);
```
Adding the neuron at layer 1 and index 31 to be observed.
```
	myNeuronObserver.addNeuronToBeObserved(1, 31);
```

Evaluating the model

Training accuracy

	myTrainer.getModelEvaluator().getTrainingAccuracy();

Testing accuracy

   	myTrainer.getModelEvaluator().getTrainingAccuracy();

Confusion Matrix

	myTrainer.getModelEvaluator().printConfusionMatrix();

XAI

xai algorithm for relevance propagation.
```
	myTrainer.relevancePropagate(2, 3);	
```

xai algotithm for most significant input neurons

	myTrainer.forwardPropagatewithExclusionInputLayerOnKSamples(2);

Observable methods

In ANN4j, every neuron is an object of its own. Every Neuron can be observed by the NeuronObserver class when the values are updated. NeuronObserver class can be extended as per the requirement of the parameters to be observed. Neurons objects can also be obtined and observed independantly.

Get a neuron object from a layer.

	myTrainer.getLayerManager().getLayer(layerNum).getNeuron(neuronNum));

Get activation of a neuron
```
	neuron.getActivation();
```
Get bias of the neuron
```
	neuron.getBias();
```
Get arraylist of the left or right connections of the neuron
```
neuron.leftConnections;
neuron.rightConnections;
```
Get weight of a connection
```
	connection.getWeight();
```

Data format

Default format

The default format for the package is MNIST format.

File type CSV consisting of the following

1 Label (Expected number)
n pixel weights n must match number of input neurons.

Example

2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,116,125,171,255,255,150,93,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,169,253,253,253,253,253,253,218,30,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,169,253,253,253,213,142,176,253,253,122,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,52,250,253,210,32,12,0,6,206,253,140,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,77,251,210,25,0,0,0,122,248,253,65,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,31,18,0,0,0,0,209,253,253,65,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,117,247,253,198,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,76,247,253,231,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,253,253,144,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,176,246,253,159,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,25,234,253,233,35,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,198,253,253,141,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,78,248,253,189,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,19,200,253,253,141,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,134,253,253,173,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,248,253,253,25,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,248,253,253,43,20,20,20,20,5,0,5,20,20,37,150,150,150,147,10,0,0,0,0,0,0,0,0,0,248,253,253,253,253,253,253,253,168,143,166,253,253,253,253,253,253,253,123,0,0,0,0,0,0,0,0,0,174,253,253,253,253,253,253,253,253,253,253,253,249,247,247,169,117,117,57,0,0,0,0,0,0,0,0,0,0,118,123,123,123,166,253,253,253,155,123,123,41,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0

Here the image 2 is represented as an array of 28*28 pixels each value represents pixel activation.

References

File rendering for other formats

ANN4j provides functionality to extend the InputFileReader to add file handling for various types of datasets apart from mnist type files. InputFileReader or MNISTFileReader can be extended by making relevant changes in file reading functions.

Constructor

The new file reader class must pass the filename to the super constructor.

    super(filename);

Getting input

The next() method is responsible for reading new line input from the dataset. It must also act as a super setter method. It must set all values like label , expectedOutputArray and inputArray

    public void next()

Getting label

This method must return the label (expected value of prediction).

    public double getLabel()

Getting Input Neuron values

This method must return the values of neurons (expected value of prediction). Example for digit recognition fo digit two, the arraylist must contain 784 elements of the pixel values.

0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,116,125,171,255,255,150,93,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,169,253,253,253,253,253,253,218,30,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,169,253,253,253,213,142,176,253,253,122,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,52,250,253,210,32,12,0,6,206,253,140,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,77,251,210,25,0,0,0,122,248,253,65,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,31,18,0,0,0,0,209,253,253,65,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,117,247,253,198,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,76,247,253,231,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,128,253,253,144,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,176,246,253,159,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,25,234,253,233,35,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,198,253,253,141,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,78,248,253,189,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,19,200,253,253,141,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,134,253,253,173,12,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,248,253,253,25,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,248,253,253,43,20,20,20,20,5,0,5,20,20,37,150,150,150,147,10,0,0,0,0,0,0,0,0,0,248,253,253,253,253,253,253,253,168,143,166,253,253,253,253,253,253,253,123,0,0,0,0,0,0,0,0,0,174,253,253,253,253,253,253,253,253,253,253,253,249,247,247,169,117,117,57,0,0,0,0,0,0,0,0,0,0,118,123,123,123,166,253,253,253,155,123,123,41,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0

    public ArrayList<Double> getInputArray()

Getting Output Neuron values

This method must return the expected values of output neurons (expected value of prediction). Example for digit recognition fo digit two, the arraylist can be 0,0,1,0,0,0,0,0,0,0

This is dependant on the model and is a design decision.

    public ArrayList<Double> getExpectedOutputArray()

Getting prediction from the output neurons

This method is used for obtaining the prediction value from the activations in the output neurons. In the digit recognitoin case as every input is mapped with same neuron it is the same. For example if neuron number 3 fires the highest, the model has predicted 3. But this needs to be overridden for different model configurations.

    public double getPredictionFromNeuronNum(int mostSignificantNeuronNumAsPrediction)

Note-

Predicted neuron and prediction are different.

Predicted neuron is the neuron which is most significant in firing. The prediction is the value corresponding to that neuron.

Example Consider case of handwritten letters database. If the neuron 4 is most significant (glows brightest) and it corresponds to label D then the predicted neuron is 4 and prediction is D.

getMostSignificantNeuronNumAsPrediction() is a method in LayerManager class which helps to get the value of the neuron which fires the most.

Restarting the file

Creates a new instance of the file reader and starts all over again.

    public void restart()

Examples

Example code https://github.com/Aatmaj-Zephyr/ANN4j/blob/main/Main.java
Sample output https://github.com/Aatmaj-Zephyr/ANN4j/blob/3721148ec24371bf095e1394fe39fc471f391466/output.txt
Sample output https://github.com/Aatmaj-Zephyr/ANN4j/blob/ef0f34b505e6e6316f94b5a660b9ef651582667d/output.txt

Other resources

More about Artificial Neural Networks https://www.3blue1brown.com/topics/neural-networks
Relevance propagation example https://towardsdatascience.com/indepth-layer-wise-relevance-propagation-340f95deb1ea
Rectification functions https://www.quora.com/What-is-the-purpose-of-rectifier-functions-in-neural-networks

ANN4j Community

Raising an issue

Please feel free to suggest any changes or point out any errors by raising an issue here

Asking for help

For asking for clarification on any topic, raise an question issue here

Community

Contributing

Please read the contributing guidelines here. Everyone is free to contribute to this project.

License

License notice]

MIT License

Copyright (c) 2022 Aatmaj

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Thank you For visiting. Please share your love by starring this repository and following me!

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!