Neural Networks – Introduction

Hi fellows, after some time, I’ve get all things running now so I can start things out. The first topic I will present is a series of articles about neural networks. I will not focus on a serious posting about it, I will try to keep as simple as possible. Also, I will draw some useful pictures or show myself thoughts on pictures to make it more comprehensive. Now, let’s start with an introduction on neural networks, showing how they work.

First, I need you to remember how our nervous system works, especially our brain. We have a special cell there, the neuron, and we have a special process which makes the neurons communicate each other, the synapse. I will explain them here, so, don’t be afraid if you don’t remember.

Just think with me about a neuron, maybe you remember how is the form of a neuron, by I will imagine and you can see my imagination on picture above:



All right, this is a neuron! Here we have the axon, the soma and the dendrites. The neuron works receiving an electric impulse on dendrites terminations, this impulse travels to soma which kind of decides if this impulse must be sent to axon or not by generating an action potential. The soma has an activation threshold potential which the impulse must break in order to be transmitted to axon terminations. If the impulse has a potential high enough to generate an action potential, it can go forward. Simple, right? Well, this process is not that simple, I know. It is far more complicated, especially when we talk about action potentials and activation threshold potentials, but it is a good way to explain how the information is transmitted. The essential here is the electric impulse part. In our brain, all the data is transmitted in form of electricity. But how we can decode it to our digital world?

Let’s think a little bit. In our electric impulse, we have a potential, a measure of voltage right? But we don’t have data. At neurons, this process is sort of made by neurotransmitters, transfering those potentials through dendrites and axons terminations during synapses, I will talk about this process later. Now we need to work with a model to match the neuron. We cannot process the data itself, so that is the idea, we will learn with the data. We will not change the input data, but we will learn with it. The neuron act in the same way by using the action potentials. The data is the stimuli from outside, this is translated as a electric potential. We can do the same defining a weight for each input. A weight is a number which means something regarding the data input. This weight can be modified as long the input goes forward the artificial neuron. This weight can also be transmitted through connected neurons and changed by them, which we will see soon. As well in neuron, the artificial neuron also needs a way to sum up those weights in order to check if this sum can be transmitted out or not, so we define two functions for it. I think I confuse you now, saying a lot of things at the same time, right? Above is a picture of what I said:

artificial neuron


On picture above, you can see a vector x (x0,x1,..) and associated to it a vector w (w0,w1,..) . The vector x is our input and vector w is the input weights. We can start with some pre-defined weights or random weights here. In our artificial neuron, the first thing we have is our sum function which will apply a sum on vector w.x (we need to multiply the two vector in order to weight vector reflect input data behavior, I will talk more about this later), giving a number W, which is transferred to our activation function or thresholding function k. The output of neuron is k(W). It is common the activation function returns a binary output (0 or 1) but it is not the rule.

But a single neuron will not do much work for us, neither in a biological neural system. In our brain we have a vast network of connected networks of neurons. Each of them transferring information and making things working in our body. Across each neuron we have a process called synapse.


NN corrected


In a neuron connection we have the following: neuron dendrites linked on other neurons axons terminations. Between this link, we do not have physical contact between structures. Between them, works the synapse process. When the action potential comes out from axon, substances called neurotransmitters acting between the terminations in order to transfer the potential from axon to dendrites termination, initiating the neuron processing on other side. There is a very complex mechanic on neurotransmitter action on potentials but I will simplify stuffs here, we will only need to know about how the potential is transferred. For our artificial neural network, we can simplify and just connect the output of a neuron as the input of a new neuron. Now we have our “neurotransmitter process”, we can think about a neurotransmitter in our brain as a kind of modifier of the potential data. We do the same thing in our artificial neurons. As soon the input come into a neuron, we multiply it with the weights on neuron (as we did above with w.x), giving us a new weight vector with information of neuron weights but with input characteristics. This process permits our artificial neural network to learn and pass this learning across neurons on network, as our brain does.


Above a simple neural network with a hidden layer. I will explain the layers of a neural network on next article and show how to implement a simple neural network, the perceptron network. I will show some implementation in python as well letting you see the network working!


See you layer!

Hello IoT World: Using IBM Watson IoT Platform with TI SimpleLink SensorTag

Hello folks! Today I will introduce you into the Internet of Things world. I am walking around a while on the beginning of the IoT thing since I was a hobbyist, but only in 2010 to 2012 I could access real Ethernet ready devices. The problem was much of the Internet part must to be done by ourselves, not having a good and reliable service to get the data. In that time, I worked with Arduino, so I was very limited with how can I collect and send those data through the cloud. Today, with more powerful and small processors, much more embedded solutions arrived to the hands of hobbyists and enthusiasts. Even Arduino has boards with powerful ARM processors nowadays. Now, with small board computers like Raspberry PI and Beaglebone Black, we have an entire computer on the palm of a hand.

And with all these new toys, the IoT world started to run and go around. Small applications such using a connected and embedded device to control your home over Internet became now bigger applications such industrial monitoring as an example. Good projects are all around to get data from devices, provide organized data for analytics, and so one. These kind of projects grows every day. Not only software is being built but new simple hardware is also being built also. Simple devices, already made to connect into smartphones, or made to connect directly into cloud to transmit data are available on the market, making the IoT world even bigger.

Today I will present two of those new stuffs. One is a new hardware from Texas Instrument, the SimpleLink SensorTag. Texas Instrument is known to build processors, sensors and a plethora of electronic hardware parts for industry, but it is also known to sell great developer kits to spread their technology across the world. They support the BeagleBoard project and sold cool kits like the old Metawatch development kit, one of the first smartwatch ever made and available for programming.

Read more

Starting over…

This is the new version of the blog, with new content and new focus. Now this is a place for my thoughts and work on topics I research. Also I will write some useful stuff on these subjects as well. My intent is not to write full tutorials of everything, but somethings will deserve a good tutorial of course. In most of times, some skill will need to understand some articles here but I will warn about this if needed.

As I am moving things on here, changing the focus and even changing the language of blog, I will make a brief of myself here. I am a fox with a mind which don’t stop thinking. So I work on many things at the same time trying to understand the world, well, most part of it. My focus is for geek stuffs, so I am fascinated about technology and science. Since I was a little, I work with computers so I always stay inside this world. Ok, I’m not a fox in real, you can read a full description of myself at the About page. But let’s consider myself as the Geek Fox, right? More precisely, Mr. Geek Fox.  I’m interested in many things, but to reach some focus here (something I don’t have a good control), I will restrain on some topics.

First of all is Artificial Intelligence. This topic takes my attention since my childhood and I always try to learn something about it. It takes a long time to learn something useful about it but I made it someday (I remember the first day I read about neural networks and see the computational representation of a neuron, in the time I don’t understand anything, neither what an activation function was, but it was fun play with it). Now, I’m trying to make this knowledge useful for something.

Internet of Things, it is a famous topic now, but for me is a little old. I start learning about it in middle of 2010, and in that time I already know the topic is already in the mainstream on engineers and hobbyists. I am a garage engineer wannabe for a long time, but I stop to do things for some years. Now I started to play around with some stuffs again.

Game development is one of my jobs now. I have my main job but I have my game project to develop. This project will have your own blog and site, so here I will post eventually some good stuffs about the subject. Also, sometimes I will tell one thing or other about my game project until the blog and site don’t get ready.

And finally, technology and other geek stuffs. As a geek, I love technology and science, so I will post my analysis on discovers of science eventually, as well good stuffs from new technologies.

That’s it fellow folks! See you around here, or on Facebook (you can get there by clicking on top of the page)