Merge pull request #638 from microsoft/staging

Syncing master <-> staging branches

Author: PatrickBue

.github/PULL_REQUEST_TEMPLATE.md

@@ -11,7 +11,8 @@
 <!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
 <!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
 - [ ] I have followed the [contribution guidelines](../CONTRIBUTING.md) and code style for this project.
+- [ ] This branch is created from `staging` and not `master`.
+- [ ] This PR is being made to `staging` and not `master`.
+- [ ] I will squash merge this PR into `staging`.
 - [ ] I have added tests covering my contributions.
 - [ ] I have updated the documentation accordingly.
-- [ ] This PR is being made to `staging` and not `master`
-- [ ] I will squash merge this PR into `staging`

NOTICE.txt

@@ -525,4 +525,31 @@ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.
+SOFTWARE.
+
+--
+
+https://github.com/ifzhang/FairMOT
+
+
+MIT License
+
+Copyright (c) 2020 YifuZhang
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -5,7 +5,7 @@
 
 In recent years, we've see an extra-ordinary growth in Computer Vision, with applications in face recognition, image understanding, search, drones, mapping, semi-autonomous and autonomous vehicles. A key part to many of these applications are visual recognition tasks such as image classification, object detection and image similarity.
 
-This repository provides examples and best practice guidelines for building computer vision systems. The goal of this repository is to build a comprehensive set of tools and examples that leverage recent advances in Computer Vision algorithms, neural architectures, and operationalizing such systems. Rather than creating implementions from scratch, we draw from existing state-of-the-art libraries and build additional utility around loading image data, optimizing and evaluating models, and scaling up to the cloud. In addition, having worked in this space for many years, we aim to answer common questions, point out frequently observed pitfalls, and show how to use the cloud for training and deployment.
+This repository provides examples and best practice guidelines for building computer vision systems. The goal of this repository is to build a comprehensive set of tools and examples that leverage recent advances in Computer Vision algorithms, neural architectures, and operationalizing such systems. Rather than creating implementations from scratch, we draw from existing state-of-the-art libraries and build additional utility around loading image data, optimizing and evaluating models, and scaling up to the cloud. In addition, having worked in this space for many years, we aim to answer common questions, point out frequently observed pitfalls, and show how to use the cloud for training and deployment.
 
 We hope that these examples and utilities can significantly reduce the “time to market” by simplifying the experience from defining the business problem to development of solution by orders of magnitude. In addition, the example notebooks would serve as guidelines and showcase best practices and usage of the tools in a wide variety of languages.
 
@@ -37,7 +37,7 @@ notebooks in this repo. Once your environment is setup, navigate to the
 
 Alternatively, we support Binder
 [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/PatrickBue/computervision-recipes/master?filepath=scenarios%2Fclassification%2F01_training_introduction_BINDER.ipynb)
-which makes it easy to try one of our notebooks in a web-browser simply by following this link. However, Binder is free, and as as result only comes with limited CPU compute power and without GPU support. Expect the notebook to run very slowly (this is somewhat improved by reducing image resolution to e.g. 60 pixels but at the cost of low accuracies).
+which makes it easy to try one of our notebooks in a web-browser simply by following this link. However, Binder is free, and as a result only comes with limited CPU compute power and without GPU support. Expect the notebook to run very slowly (this is somewhat improved by reducing image resolution to e.g. 60 pixels but at the cost of low accuracies).
 
 ## Scenarios
 

contrib/README.md

@@ -10,6 +10,7 @@ Each project should live in its own subdirectory ```/contrib/<project>``` and co
 |---|---|---|
 | [Crowd counting](crowd_counting) | Counting the number of people in low-crowd-density (e.g. less than 10 people) and high-crowd-density (e.g. thousands of people) scenarios. | [![Build Status](https://dev.azure.com/team-sharat/crowd-counting/_apis/build/status/lixzhang.cnt?branchName=lixzhang%2Fsubmodule-rev3)](https://dev.azure.com/team-sharat/crowd-counting/_build/latest?definitionId=49&branchName=lixzhang%2Fsubmodule-rev3)|
 | [Action Recognition with I3D](action_recognition) | Action recognition to identify video/webcam footage from what actions are performed (e.g. "running", "opening a bottle") and at what respective start/end times. Please note, that we also have a R(2+1)D implementation of action recognition that you can find under [scenarios](../sceanrios).| |
+| [Document Image Binarization](binarization) | Binarization is a technique to segment foreground from the background pixels. A simple technique for binarization is thresholding of gray-level or color document scanned images.| |
 
 ## Tools
 | Directory | Project description | Build status (optional) |

contrib/binarization/README.md

@@ -0,0 +1,17 @@
+# Binarization
+Binarization is a technique to segment foreground from the background pixels. A simple technique for binarization is thresholding of gray-level or color document scanned images.
+## At a glance
+
+This binarization technique is an improvement over Sauvola's binarization technique. In this work, we improve the existing Sauvola's binarization technique by preserving more foreground information in the binarized document-images. In order to achieve this, we introduce a confidence score for the background pixels. 
+
+### Input images
+
+<img src="./confidence_based_Sauvola_binarization/test_images/2.jpeg" width="33%"> </img>
+<img src="./confidence_based_Sauvola_binarization/test_images/10.jpeg" width="33%"> </img>
+<img src="./confidence_based_Sauvola_binarization/test_images/new1.jpg" width="33%"> </img>
+
+### Binary outputs
+
+<img src="./confidence_based_Sauvola_binarization/results/2_bin_new.png" width="33%"> </img>
+<img src="./confidence_based_Sauvola_binarization/results/10_bin_new.png" width="33%"> </img>
+<img src="./confidence_based_Sauvola_binarization/results/new1_bin_new.png" width="33%"> </img>

contrib/binarization/confidence_based_Sauvola_binarization/Modified-Sauvola_Binarization.ipynb

@@ -0,0 +1,64 @@
+'''
+@author: Soumyadeep Dey
+				email:Soumyadeep.Dey@microsoft.com
+				Date: October 2020
+				cite: https://drive.google.com/file/d/1D3CyI5vtodPJeZaD2UV5wdcaIMtkBbdZ/view?usp=sharing
+'''
+
+# importing required libraries
+import numpy as np
+import cv2
+from skimage.filters import threshold_sauvola
+
+
+def SauvolaModBinarization(image,n1=51,n2=51,k1=0.3,k2=0.3,default=True):
+    '''
+	 Binarization using Sauvola's algorithm
+		@name : SauvolaModBinarization
+	 parameters
+		@param image (numpy array of shape (3/1) of type np.uint8): color or gray scale image
+	 optional parameters
+		@param n1 (int) : window size for running sauvola during the first pass
+		@param n2 (int): window size for running sauvola during the second pass
+		@param k1 (float): k value corresponding to sauvola during the first pass
+		@param k2 (float): k value corresponding to sauvola during the second pass
+		@param default (bool) : bollean variable to set the above parameter as default. 
+
+			@param default is set to True : thus default values of the above optional parameters (n1,n2,k1,k2) are set to
+				n1 = 5 % of min(image height, image width)
+				n2 = 10 % of min(image height, image width)
+				k1 = 0.5
+				k2 = 0.5
+		Returns
+			@return A binary image of same size as @param image
+		
+		@cite https://drive.google.com/file/d/1D3CyI5vtodPJeZaD2UV5wdcaIMtkBbdZ/view?usp=sharing
+    '''
+
+    if(default):
+        n1 = int(0.05*min(image.shape[0],image.shape[1]))
+        if (n1%2==0):
+            n1 = n1+1
+        n2 = int(0.1*min(image.shape[0],image.shape[1]))
+        if (n2%2==0):
+            n2 = n2+1
+        k1 = 0.5
+        k2 = 0.5
+    if(image.ndim==3):
+        gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
+    else:
+        gray = np.copy(image)
+    T1 = threshold_sauvola(gray, window_size=n1,k=k1)
+    max_val = np.amax(gray)
+    min_val = np.amin(gray)
+    C = np.copy(T1)
+    C = C.astype(np.float32)
+    C[gray > T1] = (gray[gray > T1] - T1[gray > T1])/(max_val - T1[gray > T1])
+    C[gray <= T1] = 0
+    C = C * 255.0
+    new_in = np.copy(C.astype(np.uint8))
+    T2 = threshold_sauvola(new_in, window_size=n2,k=k2)
+    binary = np.copy(gray)
+    binary[new_in <= T2] = 0
+    binary[new_in > T2] = 255
+    return binary

contrib/binarization/confidence_based_Sauvola_binarization/ModifiedSauvola.pdf

@@ -0,0 +1,43 @@
+# Binarization
+Binarization is a technique to segment foreground from the background pixels. A simple technique for binarization is thresholding of gray-level or color document scanned images.
+## At a glance
+
+This binarization technique is an improvement over Sauvola's binarization technique. In this work, we improve the existing Sauvola's binarization technique by preserving more foreground information in the binarized document-images. In order to achieve this, we introduce a confidence score for the background pixels. 
+
+### Input images
+
+<img src="./test_images/2.jpeg" width="33%"> </img>
+<img src="./test_images/10.jpeg" width="33%"> </img>
+<img src="./test_images/new1.jpg" width="33%"> </img>
+
+### Sauvola outputs
+
+<img src="./results/2_bin_old.png" width="33%"> </img>
+<img src="./results/10_bin_old.png" width="33%"> </img>
+<img src="./results/new1_bin_old.png" width="33%"> </img>
+
+### Confidence based Sauvola outputs
+
+<img src="./results/2_bin_new.png" width="33%"> </img>
+<img src="./results/10_bin_new.png" width="33%"> </img>
+<img src="./results/new1_bin_new.png" width="33%"> </img>
+
+## Reference
+
+For details refer to this [paper](./ModifiedSauvola.pdf) 
+
+
+
+
+## Setup
+
+### Dependencies
+- python 3.7
+- numpy 1.16
+- opencv 4.2
+- skimage 0.17
+
+
+### Example
+
+Sample example of the usage of the said binarization technique can be found in this [notebook](./Modified-Sauvola_Binarization.ipynb).