The Data Analytics Platform – A Foundational View

It would be an understatement to say that the HelioCampus community likes data; it is more like we live and breathe data. From strategic meetings to quarterly presentations to implementation cycles, everything anyone speaks about at HelioCampus is backed by data. This is the same way our institutional partners use the platform for data-driven insights. If you are reading this article with the same passion and appreciation for data-informed decision making, then you are in the right place. The following outlines core foundational capabilities required for an analytics platform and subsequent blogs will provide additional entries as a reference guide to component products and services.

In order to provide a fully capable analytics platform, a foundation must first be established to ingest, store and secure data, as well as enable trusted access and across institutional users. Building a scalable, solid data analytics platform, requires a thought-out approach and a long-term commitment. It is safe to assume that everyone’s standard expectations of a good data platform would be that it provides massive number crunching query results at lightning speed, flexibility to scale up and down as needed (thank you Cloud technology!), absolute security and privacy and should give you the best bang for your buck.

Behind the scenes, it should also be easy to manage for the administrators, should not require 24/7 babysitting by an ops team, and should support open source technologies for the data engineers (after all, who doesn’t love open-source technologies!)

It is this set of core requirements that pushed HelioCampus’ data architects to develop our standard cloud-based architecture via AWS, the backbone of our analytics product offerings. AWS, cloud technologies and a combination of purposely selected open source and proprietary technologies comprising our tech stack enables us to efficiently and securely manage more than 30 production scale data warehouses with non-existent down time. As a result, our internal teams and, more importantly, our user community can focus on what matters the most – the data.

So, what exactly does it mean when someone says, ‘HelioCampus uses AWS to support data analytics’?

In the year 2021 it could have meant 175 different things because, by 2020, that’s how many products and services AWS offered. These offerings are spread across computing, storage, networking, database, analytics, application services, deployment, management, mobile, developer tools and IOT. Yes, there’s something for everyone. So where do you start, and how do you choose which set of AWS tools to use for your analytics platform?

Let’s step back a little and ask ourselves a basic question - what core capabilities are required to support a data infrastructure?

• Unlimited scalable storage: to begin with, you would need a place to store your data. AWS S3 allows you to store all the data you want and never run out of space.
• High-volume data exploration: for sophisticated analysis across integrated data sources, a lightning fast data warehouse, like AWS Redshift.
• High-performance computing: You would need managed application servers, like AWS EC2, to run your favorite BI tool and show off the results of your fancy query to others.
• Dedicated network and computing resources: You would also need the ability to make your data inaccessible from the public internet and accessible securely from remote places via the AWS Virtual Private Cloud (VPC.)
• Up-to-date data insights: for data sources that require frequent refreshes, you may want to have an ETL mechanism handy, like AWS Glue (if you want an ETL tool from within AWS ecosystem), or Python (Open-source, yay!).
• Secure protection and access to data: Virtual Private Network (VPN) access combined with AWS Identity and Access Management (IAM) provides for privileges and authenticated remote access by individual user credentials, preventing unauthorized access or use.

Needless to say, this blog merely scratches the surface when it comes to describing the range of services available in the AWS universe and how HelioCampus effectively leverages them in serving institutional partners. If you would like to learn more about AWS and how we used their services to continue to enhance our data analytics platform, don’t hesitate to reach out. Team HelioCampus will be happy to help you on your data journey.

In the meantime, continue to check our blog periodically as I will be sharing more detailed posts on each AWS tool mentioned!