Math and Computation in Neural Network

An Artificial Neural Network (ANN) is a series of algorithms that endeavors to recognize underlying relationships in a set of data through a process that mimics the way the human brain operates[1]. Neuron typically has a weight that adjusts as learning proceeds. This article focuses on the basic mathematical and computation process in the ANN.

Artificial Neuron

An artificial neuron is a mathematical function that models biological neurons. A Neuron can receive input from other neurons. The neuron inputs are weighted with and summed () them before being passed through into an activation function. Figure 1 shows the structure of an artificial neuron.

Figure 1: Artificial Neuron


ANN is a supervised learning that its operation involves a dataset. The dataset is labeled and split into two part at least, namely training and testing. We have expected output (from the label) for each input. During the training process, the parameter weights and biases are updated in order to achieve a result that is close to the labeled data. To identify model performance, we evaluate the model to the testing dataset to verify how good the trained model is. To get more detail how ANN learn, let begin with the last two neurons connection.

Figure 2: Simple Two Neuron Connection

is a neuron with an activation
is an index of layer. It is not an exponential.

In the training phase, the sample data is fed through the ANN. The outcome of the ANN is inspected and compared to the expected result (the label). The difference between the outcome and expected result is called Cost/Error/Loss. There are several cost functions can be used to evaluate the cost in analyzing model performance. One of the common ones is the quadratic cost function.

The cost for the network in Figure 2 in form of quadratic cost function is:

Eq (1)

The total cost of a feed through the ANN are:

Eq (2)

Equation (1) and (2) show that the closer the outcome of ANN to the expected result, the smaller the cost will be. The cost can be thought as a mathematical function of weights and biases . Hence, the fittest model can be achieved by minimizing the cost to find the suitable weights and biases.

More …

Domain Driven Design Short Summary

I believe there are tons of articles and books out there that discuss about Domain Driven Design (DDD for short). This post essentially is taken from my note archieved when I explored about DDD from several resources. I rewrite it in here as a refresh for myself mainly.

Domain Driven Design (DDD) is the concept that the structure and language of software code (class names, class methods, class variables) should match the business domain [1]. We don’t need to kill a mosquito with a cannon. We need to choose the fittest method in solving a problem. As well as DDD, it is not always be a fit method in solving application design. A software application has several attributes. Some of them deal with:

  • Amount of Data
  • Performance
  • Business Logic complexity
  • Tehnical Complexity.

From the fourth software attributes, DDD is the most suitable with an application that has a complex business logic. DDD is designed to tackle the complexity of business rules. The main goals of DDD are[1]:

  • Placing the project’s primary focus on the core domain and domain logic;
  • Basing complex designs on a model of the domain;
  • Initiating a creative collaboration between technical and domain experts to iteratively refine a conceptual model that addresses particular domain problems.

One of good resources about DDD is DDD course by Vladimir Khorikov. I also cited a few/snippet code from his course to make my summary about DDD more clearer to understand.

Terms in DDD

Ubiquitous Languange

Ubiquitous Languanges bridges the gab between developer and business expert/ domain expert/ subject matter expert (SME). The Ubiquitous Languange notion come up to avoid misunderstanding between them. For example, the developer has a class Product that represent business term Product (both product and package). On the other hand the business expert treat Product and Package as different things. By the condition, it is needed the same languange to avoid misunderstanding.

Bounded Context

Bounded Context is a central pattern in DDD. It is the focus of DDD’s strategic design section which is all about dealing with large models and teams. DDD deals with large models by dividing them into different Bounded Contexts and being explicit about their interrelationships [Martin Fowler - BoundedContext].

Bounded Context notion comes up to make clear boundries between different parts of the system. Let’s say our system consist of Sales and Support, we can seperate Product into Sales and Support context to make clear boundries between the two.

Figure 1: Bounded Contexts
Source: https://martinfowler.com/bliki/BoundedContext.html

Figure 1 shows that Bounded Context aims for separating the model and explicitly drawn the boundaries between pieces. The reason for context separation is that as the application grows, it becomes harder to maintain a single unified model as it becomes larger and more people get involved in the development process.

Sub Domain

The difference between Sub Domain and Bounded Context is that the Sub Domain is a Problem Space. Meanwhile Bounded Context is a Solution Space. Sub Domain and Bounded Context have a 1-to-1 relation. It means that a Sub Domain should be covered with exactly a Bounded Context.

Figure 2: Sub Domain - Bounded Contexts

Since Sub Domain is a Problem Space, it should be defined by Business Expert or Domain Experts. The Sub Domain topic usually will come up when we have a talk with the Domain Expert while gathering requirement.

Core Domain

The notion that focuses on Domain Model which is the most important part of the system.

More …

Generate POJO and Hibernate Mapping Using Hibernate Tools

Sometime we get into a condition in which we need to create POJO classes from an existing database. It could take time if the database has many tables. Instead of manually create them, we could use Hibernate Tool. Hibernate Tools is an eclipse plugin that can be installed from the eclipse marketplace.

In this post, we use Spring Tool Suite (STS) 4 instead of directly use the eclipse version of the IDE. Meanwhile the database we use is PostgreSQL. It should be applicable to any other database.

The steps of using Hibernate Tools is summarized as follow. I append many screenshots on this post in order the steps are easier to follow.

Step 1 : Install Hibernate Tools plugin

To install Hibernate Tools plugin, go to menu Help > Eclipse Marketplace .

Enter Hibernate Tools in the search field.

Follow the installation step until the installation success and finish. Eclipse will show us a pop up message to restart it once the installation finished.

More …

PPTP on Ubuntu

Previously, I have posted how to connect macOS to VPN server through PPTP protocol</a>. This post decribed how to do the same thing for Ubuntu. I used Ubuntu 18.04.3 LTS for testing.

The first step is installing PPTP client for Ubuntu.

sudo apt-get -y install pptp-linux

Create VPN configuration file

sudo nano /etc/ppp/peers/myPPTP

paste the following script

pty "pptp IP_ADDRESS --nolaunchpppd --debug"
name USERNAME
password PASSWORD
remotename PPTP
require-mppe-128
require-mschap-v2
refuse-eap
refuse-pap
refuse-chap
refuse-mschap
noauth
debug
persist
maxfail 0
defaultroute
replacedefaultroute
usepeerdns
More …

Configure PHP and Virtual Host macOS High Sierra

Apache is pre installed on macOS High Sierra. We just need to run its service with the following command to activate it.

$ sudo apachectl start

Then we just open http://localhost via browser. Apache will display a default HTML page come with it.

Activating PHP Module

High Sierra also comes with PHP 7. So we don’t need to install it manually. To activate PHP module

  1. Edit /etc/apache2/httpd.conf
  2. Uncomment / remove remark of #LoadModule php7_module libexec/apache2/libphp7.so
  3. Save it and restart apache using
         $ sudo apachectl restart
    

After applying the steps above, the php module should be activated and ready to use. In this post we will test it after configuring virtual host.

Configuring Virtual Host

The steps of configuring apache virtual host are :

  1. Enabling virtual host configuration in apache config by editing /etc/apache2/httpd.conf.

     $ sudo nano /etc/apache2/httpd.conf
    
  2. Uncomment the section Include /private/etc/apache2/extra/httpd-vhosts.conf, then save it.

  3. Create site directory. As an example in this post, let’s create a Site folder in home directory called </code>/Users/USERNAME/Sites</code>. Our website sample directory will be put in it, let’s create a directory called </code>/Users/USERNAME/Sites/neutro.io</code> and create an /Users/USERNAME/Sites/neutro.io/index.php with simple PHP syntax.

     <?php
     phpinfo();
     ?>
    
  4. Create virtual host configuration by editing the virtual host config

     $ sudo nano /etc/apache2/extra/httpd-vhosts.conf
    

    The following code is an example of virtual host with domain name neutro.io

     <VirtualHost *:80>
         ServerName neutro.io
         ServerAlias www.neutro.io
         DocumentRoot "/Users/neutro/Sites/neutro.io"
    
     <Directory /Users/neutro/Sites/neutro.io>
             Options Indexes FollowSymLinks
             #Options All Indexes FollowSymLinks
             AllowOverride None
             Require all granted
     </Directory>
    
    
         ErrorLog "/private/var/log/apache2/neutro.io-error_log"
         CustomLog "/private/var/log/apache2/neutro.io-access_log" common
         ServerAdmin web@neutro.io
     </VirtualHost>
    

    In this example, we create a neutro.io virtual host that refers to /Users/neutro/Sites/neutro.io as physical directory.

  5. Register domain for localhost

    Since we use neutro.io as domain for localhost, we need to add the domain and www alias to resolve to the localhost address by editing

        $ sudo nano /etc/hosts
    

    and add the following line

        127.0.0.1   neutro.io   www.neutro.io
    
  6. Restart apache

     $ sudo apachectl restart
    

When we open in browser http://neutro.io, we should get a page that display PHP info.

More …