Altabel Group's Blog

Archive for the ‘Big Data’ Category

If the experts’ estimates regarding IoT are correct, it means that in 5-10 years there will be more than 50 billion interconnected devices in the world. And they all will generate zettabytes of data, which can be and should be collected, organized and used for various purposes. Hence the tight correlation between IoT and Big Data is hard to ignore, because IoT and Big Data are like Romeo and Juliet – they are created for each other. The unprecedented amount of data produced by IoT would be useless without the analytic power of Big Data. Contrariwise, without the IoT, Big Data would not have the raw materials from which to model solutions that are expected of it.

What are the impacts of IoT on Big Data?

The IoT revolution means that almost every device or facility will have its own IP address and will be interconnected. They are going to generate a huge amount of data, spewing at us from different sides – household appliances, power stations, automobiles, train tracks and shipping containers etc. That’s why the companies will have to update technologies, instruments and business processes in order to be able to cope with such great amount of data, benefit from its analysis and finally gain profit. The influence of Big Data on IoT is obvious and it is conducted by various means. Let’s take a closer look at the Big Data areas impacted by IoT.

Methods and facilities of Data Storage

IoT produces a great and stable flow of data, which hits companies’ data storage. In response to this issue, many companies are shifting from their own storage framework towards the Platform as a Service (PaaS) model. It’s a cloud-based solution, which supports scalability, flexibility, compliance, and an advanced architecture, creating a possibility to store useful IoT data.

There are few options of models in the modern cloud storage: public, private and hybrid. Depending on the specific data nature, the companies should be very accurate while choosing a particular model. For instance, a private model is suitable for the companies who work with extremely sensitive data or with the information which is controlled by the government legislation. In other cases, a public or hybrid option will be a perfect fit.

Changes in Big Data technologies

While collecting the relevant data, companies need to filter out the excessive information and further protect it from getting attacked. It presupposes using highly productive mechanism that comprises particular software and custom protocols. Message Queue Telemetry Transport (MQTT) and Data Distribution Service (DDS) are two of the most widely used protocols. Both of them are able to help thousands of devices with sensors to connect with real-time machine-to-machine networks. MQTT gathers data from numerous devices and puts the data through the IT infrastructure. Otherwise, DDS scatters data across devices.

After receiving the data, the next step is to process and store it. The majority of the companies tend to install Hadoop and Hivi for Big Data storage. However there are some companies which prefer to use NoSQL document databases, as Apache CouchDB and others. Apache CouchDB is even more suitable, because it provides high throughput and very low latency.

Filtering out redundant data

One of the main challenges with Internet of Things is data management. Not all IoT data is relevant. If you don’t identify what data should be transmitted promptly, for how long it should be stored and what should be eliminated, then you could end up with a bulky pile of data which should be analyzed. Executive director of Product Marketing Management at AT&T, Mobeen Khan, says: “Some data just needs to be read and thrown away”.

The survey carried out by ParStream (an analytical platform for IoT) shows that almost 96 % of companies are striving to filter out the excessive data from their devices. Nevertheless only few of them are able to do it efficiently. Why is it happening? Below you can see the statistics, depicting the main problems which most of the companies are facing with the data analysis procedure. The percentage figure points out the percentage of the respondents to the ParStream survey confronting the challenge.

• Data collection difficulties – 36%
• Data is not captured accurately – 25%
• Slowness of data capture – 19%
• Too much data to analyze in a right way – 44%
• Data analyzing and processing means are not developed enough – 50%
• Existing business processes are not adjustable to allow efficient collection – 24%

To perform the action of filtering out the data effectively, organizations will need to update their analysis capabilities and make their IoT data collection process more productive. Cleaning data is a procedure that will become more significant to companies than ever.

Data security challenges

The IoT has made an impact on a security field and caused challenges which can’t be resolved by traditional security systems. Protecting Big Data generated from IoT arouses complications as this data comes from various devices, producing different types of data as well as different protocols.

The equally important issue is that many security specialist lack experience in providing data security for IoT. Particularly, any attack can not only threaten the data but also harm the connected device itself. And here is the dilemma when a huge amount of sensitive information is produced without the pertinent security to protect it.

There are two things that can help to prevent attacks: a multilayered security system and a thorough segmentation of the network. The companies should use software-defined networking (SDN) technologies combined with network identity and access policies for creating a dynamic network fragmentation. SDN-based network segmentation also should be used for point-to-point and point-to-multipoint coding based on the merger of some software-defined networking and public key infrastructure (SDN/PKI). In this case data security mechanisms will be keeping pace with the growth of Big Data in IoT.

IoT requires Big Data

With the emerging of IoT step by step many questions arises: Where is the data coming from IoT going to be stored? How is it going to be sorted out? Where will the analysis be conducted? Obviously, the companies which will be able to cope with these issues the next few years are going to be in prime position for both profits and influence over the evolution of our connected world. The vehicles will become smarter, more able to maintain larger amounts of data and probably able to carry out limited analytics. However as IoT grows and companies grow with IoT, they will have many more challenges to resolve.

What do you think about the evolving of Big Data in IoT? Have you already experienced the challenges of Big Data in IoT? And do you have any ideas about the progressive solutions to these challenges? I’ll be happy to hear your opinion in the comments below. Please, feel free to share your thoughts.

 

Anastasiya Zakharchuk

Anastasiya Zakharchuk

Business Development Manager || LI Profile

E-mail: anastasiya.presnetsova@altabel.com
Skype: azakharchuk1
www.altabel.com

BI

When a technical term is used more and more frequently the exact definition becomes “blurred” and its true meaning is usually greatly distorted.

This what happened to the term ‘business intelligence’ or BI. Ever since, when the term had only appeared, the development of technologies has substantially expanded our understanding of BI and of what advantage and benefit the company can retrieve from their available data.

So, what does ‘business intelligence’ mean today? How it could be useful for companies and how to apply its underlying ideas correctly to ensure the steady growth of efficiency and profitability of a business?

What is business intelligence? Why is it important?

BI consists of two completely diverse, but at the same time complementing one another aspects.

  1. Value for the business.

    Implies how companies can use the available information in order to multiply profit and efficiency and bring new products and services to the market successfully.

  2. IT strategy.

    Includes the idea of what technological solutions to apply in order to achieve greatest possible utility of BI.
    Presentation of data in a specific format for efficient usage by the company has always been a challenging task. For many organizations, it is quite complex to determine what particular information is required for a specific use.

Such business analysis requires certainty in methodologies and goals.

Earlier BI resources were limited by the lack of available data collection technologies. Nevertheless, modern technologies such as big data, analytics, mobile services and cloud computing in their combination allow obtaining a continuous flow of detailed information quite fast and with no serious investments.

Still, the current bottom line lies in extracting some valuable sense from these data and, in many respects, it is much more complicated than collecting information itself.

Five efficiency criteria of BI-system (and BI-strategy)

1. While selecting a BI-system one should be guided by the real needs of a particular company

The most common and at the same time the most dangerous mistake is when the BI-systems dictate the strategy of their usage. As a result, the company gets plenty of non-synchronized applications, awkward interface and the infrastructure that is already out of date, yet so entrenched in the IT system that could be barely substituted.

2. Be flexible

Flexible model of the integration of the appropriate software involves constant repetition of certain operations with the gradual development of the system. This allows companies to evaluate the success of the project at any point of time, to determine at what stage it is and towards what it moves.

As a rule, creating, testing and integration of BI-technologies goes much more smoothly when the company receives real-time feedbacks from all the running processes and is able to make required adjustments on the fly. It is vital for BI-systems!

3. User-friendly interface

BI-solutions focus on collection, visualization and management of the data.
Usually, when it comes to large amounts of numeric information companies face a risk to get exceptionally technical, inconvenient and incomprehensible data for the “illiterate” users of the system. This information is highly functional, but impractical, especially when it is badly integrated with other applications.

Integration is a key point in deploying BI-technologies. In case the interface is non-intuitive, complex and inconvenient for the end users, BI-system will definitely work inefficiently.

There is a tendency to allocate significant resources for the integration of the latest technologies promising unprecedented results. However, such investments potentially may do more harm than good. Intelligent, targeted and smooth integration is the key to avoid serious errors during implementation.

4. BI is a tool available to everyone

BI has been long used by completely different users, not only by experts with appropriate education and experience. BI-system should be simple and easy to understand to everyone.

For this purpose, companies have to attain the convenience of analytics and the reports drawn on its basis; it should be simple and demonstrative. The collected data should be presented in the way so that any user could easily make definite conclusions.

5. Centralize your data

The desire to achieve the result, based on useful information implies proper data handling. Receiving data from multiple sources and storing it in a centralized information DB, capable of filtering, sorting and removing the unnecessary is critical for the deployment of the applications involved into making business decisions. Apart from that, risk management also becomes more effective through transparency and structure.

General excitement over BI is evident

The role that IT plays in the world has significantly changed over the past few years thanks to the information ‘boom’. Still, construction of a technological infrastructure is not enough for successful data management.

That is why, ‘business intelligence’ it is not just a fashionable term it is a concept that demonstrates the need to move beyond the paradigm of a separate, isolated existence of data analysis and business goals.

In fact, BI reminds us that technologies and business must be closely linked, so that the business goals and business guidelines predetermine the choice of software and, the software in return would provide useful information leading business to success.

 

Tatyana Ogneva

Tatyana Ogneva
tatyana.ogneva@altabel.com
Skype ID: ognewatatyana
Business Development Manager (LI page)
Altabel Group – Professional Software Development

The stumbling block for many companies and the reason why organizations fall behind in the planning and pre-planning stages of big data, appears to be confusion on how best to make big data work for the company and pay off competitively.

With all the talk about rapid deployment and breakneck business change, there can be a tendency to assume that businesses are up and running with new technologies as soon as these technologies emerge from proof of concept and enter a mature and commercialized state. However, the realities of where companies are don’t always reflect this.

Take virtualization. It has been on the scene for over a decade-yet recent research by 451 Research shows that only 51 percent of servers in enterprise data centers around the world are virtualized. Other recent survey data collected by DataCore shows that 80 percent of companies are not using cloud storage, although cloud concepts have also been with us for a number of years.

This situation is no different for big data, as reflected in a Big Data Work Study conducted by IBM’s Institute of Business Value. The study revealed that while 33 percent of large enterprises and 28 percent of mid-sized businesses have big data pilot projects under way, 49 percent of large enterprises and 48 percent of mid-sized businesses are still in big data planning stages, and another 18 percent of large enterprises and 38 percent of mid-sized businesses haven’t yet started big data initiatives.

The good news is that the study also showed that of those organizations actively using big data analytics in their businesses, 63 percent said that the use of information and analytics, including big data, is creating a competitive advantage for their organization–up from 37 percent just two years earlier.

The stumbling block for many and the reason why organizations fall behind in the planning and pre-planning stages of big data, appears to be confusion on how best to make big data work for the company and pay off competitively.

Big data projects need to demonstrate value quickly and be tightly linked to bottom line concerns of the business if big data is to cement itself as a long-term business strategy.

In far too many cases when people plan to build out a complete system and architecture before using a single insight or building even one predictive model to accelerate revenue growth. Everyone anticipates the day when Big Data can become a factory spitting out models that finally divulge all manner of secrets, insights, and profits.

So how do you jump start your big data efforts?

Find big data champions in the end business and business cases that are tightly constructed and offer opportunities where analytics can be quickly put to use.

When Yarra Trams of Melbourne Australia wanted to reduce the amount of repair time in the field for train tracks, it placed Internet sensors over physical track and polled signals from these devices into an analytics program that could assess which areas of track had the most wear, and likely would be in need of repair soon. The program reduced mean time to repair (MTTR) for service crews because it was able to preempt problems from occurring in the first place. Worn track could now be repaired or replaced before it ever became a problem-resulting in better service (and higher satisfaction) for consumers.

Define big data use cases that can either build revenue or contribute to the bottom line.

Santam, the largest short-term insurance provider in South Africa, used big data and advanced analytics to collect data about incoming claims, automatically assessing each one against different factors to help identify patterns of fraud to save millions in fraudulent insurance payments.

Focus on customers

There already is a body of mature big data applications that surround the online customer experience. Companies (especially if they are in retail) can take advantage of this if they team with a strong systems integrator or a big data products purveyor with experience in this area.

Walmart and Amazon analyze customer buying and Web browsing patterns for help in predicting sales volumes, managing inventory and determining pricing.

Kristina Kozlova

Kristina Kozlova
Kristina.Kozlova@altabel.com
Skype ID: kristinakozlova 
Marketing Manager (LI page)
Altabel Group – Professional Software Development

WHAT

In today’s business and technology world you can’t have a conversation without touching upon the issue of big data. Some would say big data is a buzzword and the topic is not new at all. Still from my point of view recently, for the last two-three years, the reality around the data has been changing considerably and so it makes sense to discuss big data so hotly. And the figures prove it.

IBM reports we create 2.5 quintillion bytes of data every day. In 2011 our global output of data was estimated at 1.8 billion terabytes. What impresses it that 90 percent of the data in the world today was created in the past two years according to Big Blue. In the information century those who own the data and can analyze it properly and then use it for decision-making purpose will definitely rule the world. But if you don’t have the tools to manage and perform analytics on that never-ending flood of data, it’s essentially garbage.

Big data is not really a new technology, but a term used for a handful of technologies: analytics, in-memory databases, NoSQL databases, Hadoop. They are sometimes used together, sometimes not. While some of these technologies have been around for a decade or more, a lot of pieces are coming together to make big data the hot thing.

Big data is so hot and is changing things for the following reasons:
– It can handle massive amounts of all sorts of information, from structured, machine-friendly information in rows and columns toward the more human-friendly, unstructured data from sensors, transaction records, images, audios and videos, social media posts, logs, wikis, e-mails and documents,
– It works fast, almost instantly,
– It is affordable because it uses ordinary low-cost hardware.

WHY NOW

Big data is possible now because other technologies are fueling it:
-Cloud provides affordable access to a massive amount of computing power and to loads of storage: you don’t have to buy a mainframe and a data center, and pay just for what you use.
-Social media allows everyone to create and consume a lot of interesting data.
-Smartphones with GPS offer lots of new insights into what people are doing and where.
-Broadband wireless networks mean people can stay connected almost everywhere and all the time.

HOW

The majority of organizations today are making the transition to a data-driven culture that leverages data and analytics to increase revenue and improve efficiency. For this a complex approach should be taken, so called MORE approach as Avanade recommends:
-Merge: to squeeze the value out of your data, you need to merge data from multiple sources, like structured data from your CRM and unstructured data from social news feeds to gain a more holistic view on the point. The challenge here is in understanding which data to bring together to provide the actionable intelligence.
-Optimize: not all data is good data, and if you start with bad data, with data-driven approach you’ll just be making bad decisions faster. You should identify, select and capture the optimal data set to make the decisions. This involves framing the right questions and utilizing the right tools and processes.
-Respond: just having data does mean acting on it. You need to have the proper reporting tools in place to surface the right information to the people who need it, and those people then need the processes and tools to take action on their insights.
-Empower: data can’t be locked in silos, and you need to train your staff to recognize and act on big data insights.

And what is big data for your company? Why do you use it? And how do you approach a data-driven decision-making model in your organization?

Would be interesting to hear your point.

Helen Boyarchuk

Helen Boyarchuk
Helen.Boyarchuk@altabel.com
Skype ID: helen_boyarchuk
Business Development Manager (LI page)
Altabel Group – Professional Software Development


%d bloggers like this: