Understanding Web Services

Today I was invited to a short presentation at Embedded Systems and Pervasive Computing Lab – UFCG for explaining how web services work.

You can find the presentation bellow:

 

 

Designing Scalable Applications

Today I was invited to a short talk with students of Software Design by Tiago Massoni, D.Sc., associate professor of Federal University of Campina Grande – UFCG. The discussion was about some practices adopted for software scalability and performance. Certainly it can bring to you good ideias for handling the growing of your application.

Software Testability in IoT / Embedded Programming

In this article I will explain some testing strategies for IoT solutions using devices like:  ARDUINO, Personal Medical Devices (PHD) and mobile phones.

Introduction

Some years ago I had an amazing experience while working as part of  the team challenged to create an open-source software implementation of the IEEE 11073 protocol, and at that moment, it  seemed to be the only open-source implementation of this standard. The protocol specifies how to exchange data for personal health devices (PHD) and it covers a huge kind of “health & fitness” devices like: pedometers, heart rate monitors, etc. The implementation should cover many different types of data format, communication flows and device specializations (families), and it does not rely on a single transport layer such as TCP/IP, BLE, Bluetooth LE, etc.

The Challenge

Update a device firmware or a protocol stack at operating system is a critical operation, you should do it only when needed, sometimes it is simply not an option to upgrade the firmware of so many devices in production.

At that time there were a few kind of devices available supporting just a small part of the protocol, so we had to create a strong testing strategy to keep the solution compatible with future releases. Otherwise, our product would not have the proper testing coverage, also each change we made to update the software later could potentially break the functionality for some of the target platforms already in use.

To mitigate this risk I recommended some development practices such as: Continuous Integration and different levels of automated testing, but for such low level programing (in C language) it sounded crazy at first time. However, we achieved interesting results.

Designing the testing strategy for your solution

1) Isolate the CORE and the INTEGRATION code

I recommend for IoT and embedded programs to use a Intelligent Software Agent design, such as Simple Reflex Agent (see picture)

IntelligentAgent-SimpleReflex

The inputs are represented by sensors data and the reactions may be the state changes and output signals.

For this scenario the CORE is the reactive part of your application, it is the high-level abstraction responsible to respond to an INPUT and to produce an effect. If you isolate this logic you can validate it on multiple environments (development, testing…)  and platforms (PC, Mobile, etc.)  without rely on how the INPUT is generated (a sensor data, a hardware interruption, a network packet, a REST request or response). The input is dependent on the environment, so you can keep in the INTEGRATION code the control of data input.

2) Choose the testing strategy for each level

You can use regular unit tests to cover the CORE functions. I recommend to create pre-defined test scenarios that can be used in many test cases. For instance, I created some data files with the test input and the expected output test data to be compared with the real test output.

The integration area is where the things get complex, because it would be needed to use the real environment.

For this kind of application I opted for manual procedures when performing end-to-end testing with real devices. When real devices were not suitable, we created test agents at transport layer level (network, Bluetooth, etc.) to send the expected data units.

3) Validate it using a Continuous Testing process
typical_pyramid

The recommend test distribution, i.e., the quantity of tests by test categories, to achieve quality with a reasonable effort is to have cheaper tests to be maintained in automated test process.

Unit tests are simple and easy to maintain when comparing with other test level. So, as soon as you have unit tests enabled for you project you can enable a test suite to run in continuous integration server to validate that there is no bad side-effect for each new code change.

Final comments

What about other IoT gadgets like ARDUINO? You can use similar approach also. However, when your device does not have a LCD display, storage or network communication, it is hard to verify program output and you cannot use unit tests.

For this cases I use alternatives for system testing, like:

  1. Write the program output to serial port for debugging the data using a cable connected to the device.
  2.  Create a self-testing procedure on your app initialization. It is very common that this kind of embedded programs provide some checks during initialization on device, emitting signals like blinking lights or sounding beeps to notify about anything that breaks the standard functioning. For a recent project, I created an self-testing mode for an ARDUINO program to make it show one different agent reaction each 5 seconds, 30 seconds later I could validate all agent reactions for all expected scenarios in the sequence.
  3. Do not let protocol be a restriction, use intermediate agents. When the device was not able to communicate directly to the server (using TCP, HTTP, etc.), you can use other devices like mobile phones  and applications to gather the information using NFC, Bluetooth, or other protocol needed.

I hope this information could be useful for you. If you want to find more about the open-source project described here go to https://github.com/signove/antidote.

Unfortunately, my contribution history was lost in the  migration process to github, but I have to thank the owners because they kept my name in source code.

You can see the agent code in: agent.c

Book Review: Building a RESTful Web Service with Spring

I am happy to announce the publishing of this book. I have had the pleasure of working with technical team that produced it, and I can say for sure it will be a good acquisition for those who want to learn and improve the expertise in designing and implement RESTful services in high professional business scenarios.

This book is a very practical guide to create your first RESTFul web service, or even to improve your current web product. It covers major topics regarding architectural design, testing and useful performance aspects for running your service in a high scalable manner.

There are also good code samples to demonstrate the chapter topics, all based in the Spring Framework, a great development platform to rapid java application development.

Book information

Author: Ludovic Dewailly

Pages: 115

You can get it at a good price for Kindle, or buy it directly from packtpub web site here.

_________________________________________________________________

Versão em português (pt-BR version)

Estou feliz em anunciar a publicação deste livro. Tive o prazer de trabalhar na equipe técnica que o produziu e posso dizer com certeza que será uma boa aquisição para aqueles que querem aprender e aperfeiçoar sua habilidade em projetar e implementar serviços RESTful em cenários de negócio altamente profissionais.

Esse livro é um guia prático para criar o seu primeiro RESTful web service, ou até melhorar seu projeto existente. Ele aborda os principais tópicos relacionados a arquitetura, testes e aspectos de performance para executar seu serviço de uma maneira extremamente escalável.

Há também bons exemplos de código para demonstrar os capítulos, tudo baseado no Spring Framework, uma ótima plataforma de desenvolvimento para  rápida implementação de aplicações java.

Hexagonal Architecture – Java Starter Project

I have shared the sample code for the  Hexagonal Architecture presentation. You can find the source code  at: github.com/fabricioepa/lab-hexagon-java But I should remeber you these patterns are not exclusively related to the Java technologies and frameworks.

My professional experience using this technique

It was very simple to create a decoupled REST adapter from the core application, it should be simple to create another kind of adapter too. However, if the application domain is quite BIG or there are many ports and adapters, you will have some overhead to maintain different domains translated from the original application core domain. The PORT interface should also have a stable API definition, you should design it to support evolution without easily break compatibility with the adapters implementations for that PORT. As inital strategy to achieve that design, the project has domain events to encapsulate all input and output data for the service, enabling the application to evolute without directly break the adapter code of the API.

core/api/src/main/java/com/ticketapp/core/api/TicketService.java

/**
 * Ticket Service - Use Case API
 */
public interface TicketService {

    TicketsReadEvent list(ReadTicketsEvent event);

    TicketCreatedEvent create(CreateTicketEvent ticket);

    TicketUpdatedEvent update(UpdateTicketEvent ticket);

    TicketDeletedEvent delete(DeleteTicketEvent ticket);
}

This is one reason for this pattern has often been applied to microservices architecture, because it works nice for multiple system integrations and also for small/medium sized application domains. If the application domain starts to grow up, may be it is the time to think about design a new hexagon to it.

Feel free to bring up your ideas 😉

Hexagonal architecture

Objective

“Allow an application to equally be driven by users, programs, automated test or batch scripts, and to be developed and tested in isolation from its eventual run-time devices and databases.” (Alistair Cockburn)

Summary

This approach aims to simplify the application integration and it is a powerful strategy to rapidly compose new services. The pattern has been often applied with the microservice architecture and may be the future of the SOA, bypassing the meddiation of the Enterprise Service Bus (ESB), enabling high performance and scalability.