Some years ago, an investment to analyze data was quite expensive in terms of hardware, networking, knowledge and skills usually external to the organization and obviously data center facilities. Nowadays you can enjoy cloud native data analytics tools that can be deployed in minutes in any region of the world. This cutting edge technologies are evolving to work better together as evolves a music orchestra when musicians and the conductor know each other better. He can give then a splendid performance in the concert. So happens in the cloud, the maturity of the native tools lets you decouple components so that they run and scale independently.

But why Big Data on prem is called to extinction?. Well, it is a matter of being cost-effective in middle-terms. There are some factors that have a great impact on CIOs and CFOs to change their minds:

Big Data on premise is rigid and inelastic as the capacity planning done by the architects to build those solutions is based on picks and needs to take into account the worst cases in performance. They can not scale on demand and if you need more resources you have to wait till they are available even weeks. On the other hand, you have a technical debt if you are underutilize your Big data infrastructure.

Big Data and Data analytics platforms on premise requires a lot skills and knowledge in place from Storage, to networking, from data engineering to data science. It is complex to maintain and upgrade. What is prone to failures and low productivity.

Data and AI&ML live in separate worlds in an on premise infrastructure. Two silos that you need to interconnect. Something that doesn’t happen on the cloud.

---

Move to the next level. Azure Synapse

Azure Synapse is a whole orchestra prepare to give a splendid performance in the concert. It is the evolution of Azure Data Warehouse as joins enterprise data warehousing with Big Data analytics.

It unifies data ingestion, preparation & transformation of data . So companies can combine and serve enterprise data on-demand for BI and AI/ML. It supports two types of analytics runtimes – SQL and Spark based that can process data in a batch, streaming, and interactive manner. For a Data Science is great because supports a number of languages like SQL, Python, .NET, Java, Scala, and R that are typically used by analytic workloads. You don’t have to worry for escalation, you has a virtually unlimited scale to support analytics workloads.

Deploy Azure Synapse in minutes – Using Azure Quick-Start templates it is possible to deploy your data analytics platform in minutes..choose 201-sql-data-warehouse-transparent-encryption-create to do so synchronize with your Repo on Azure devops and start to configure your deployment strategy.

Ingesting and Processing Data enhacements- Data from several origins can be load to the SQL pool component on Azure Synapse. Let’s say the old data warehouse. To load that data we can use a storage account or even better a data lake storage with the help of polybase, we can use other Azure component called Azure Data factory to bring data from several origins or traditional ones like BCP for those working with SQL. After cleaning the data on staging tables you can proceed to copy to production all that make senses.

A great advantage is that you can now get rich insights on your operational data in near real-time, using Azure Synapse Link. ETL-based systems tend to have higher latency for analyzing your operational data, due to many layers needed to extract, transform and load the operational data. With native integration of Azure Cosmos DB analytical store with Azure Synapse Analytics, you can analyze operational data in near real-time enabling new business scenarios.

Querying Data – You can uses Massive Parallel Processing (MPP) to run queries across petabytes of data quickly. Data Engineers can use the familiar Transact-SQL to query the contents of a data warehouse in Azure Synapse Analytics as well as developers can use Python, Scala and R against the Spark engine. There is also support for .Net and Java.

Moreover now it is possible to query on demand…

Authentication and Security – Azure Synapse Analytics supports both SQL Server authentication as well as Azure Active Directory. Also you can configure a RBAC strategy to access data with less privileged principals.

Finally, even you can implement MFA to protect your data and operational work.

In the next post, i will show you how work other pieces and components of Data Cloud solutions and the great benefits they bring in cost-savings and technical advantages..

See you them…