Tag: Python
-
How to orchestrate an ETL Data Pipeline with Apache Airflow
Posted on March 10, 2023, Level intermediate Resource Length medium
Data Orchestration involves using different tools and technologies together to extract, transform, and load (ETL) data from multiple sources into a central repository. By Aviator Ifeanyichukwu.
Tags apache database nosql data-science python big-data
-
Temporal graph neural networks with Pytorch - How to create a simple recommendation engine on an Amazon dataset
Posted on February 11, 2023, Level beginner Resource Length long
Over the course of the last few months, we at Memgraph have been working on something that we believe could be helpful with classical graph prediction tasks. With our latest newborn query module, you will have the option of performing both label classification and link prediction. By Antonio Filipovic.
Tags cloud big-data devops data-science python
-
8 most popular Python HTML web scraping packages with benchmarks
Posted on February 4, 2023, Level intermediate Resource Length long
This blog post will cover Python web scraping packages in terms of their speed, ease of use, and personal investigations. This blog post won't cover what webscraping is and how parsers work. By Dmitriy Zub.
Tags python programming web-development app-development performance
-
How to build a GraphQL server in Python with Graphene
Posted on December 4, 2022, Level intermediate Resource Length medium
Facebook developed GraphQL during their quest to find a better and more flexible way to work across their data. They found that they could achieve this flexibility with a query-based approach. By defining a schema that the queries can run against, an API based on GraphQL enables clients to make fewer requests while still being able to work with as much (or as little) of the data that's exposed through the query as desired. By Vince Power.
Tags apis restful python app-development web-development
-
Asynchronous tasks with Flask and Celery
Posted on October 23, 2022, Level intermediate Resource Length medium
If a long-running process is part of your application's workflow, rather than blocking the response, you should handle it in the background, outside the normal request/response flow. By Michael Herman.
Tags python web-development app-development
-
PyTorch on Google Cloud: How to train PyTorch models on AI Platform
Posted on September 23, 2022, Level intermediate Resource Length long
PyTorch has been predominantly used in research and in recent years it has gained tremendous traction in the industry as well due to its ease of use and deployment. By Rajesh Thallam, Vaibhav Singh.
Tags gcp google data-science python json
-
Guide to Reinforcement Learning with Python and TensorFlow
Posted on July 3, 2022, Level intermediate Resource Length long
In reinforcement learning, self-learning agent learns how to interact with the environment and solve a problem within it. In this article, we present complete guide to reinforcemen learning and one type of it Q-Learning (which with the help of deep learning become Deep Q-Learning). We learn about the inspiration behind this type of learning and implement it with Python, TensorFlow and TensorFlow Agents. By Nikola M. Zivkovic.
Tags cio learning big-data data-science python
-
Automating AWS IAM remediation in Python
Posted on June 20, 2022, Level advanced Resource Length long
Since automating IAM remediation requires recurring parsing of JSON documents, modulating your parsing function(s) will save you time as you work on each control. If you have controls that pertain to trust policies, you will need a separate function to parse those out since the format differs from that of permission policies. By Cody Bench.
Tags python app-development infosec programming cio
-
Introduction to Linear Programming in Python
Posted on June 19, 2022, Level intermediate Resource Length medium
A guide to mathematical optimization with Google OR-Tools. Linear programming is a technique to optimize any problem with multiple variables and constraints. It's a simple but powerful tool every data scientist should master. By Maxime Labonne.
Tags python app-development programming open-source
-
How to make anonymous requests using TorRequests and Python
Posted on April 11, 2022, Level intermediate Resource Length short
Tor is quite useful when you have to use requests without revealing your IP address, especially when you are web scraping. This tutorial will use a wrapper in python that helps you with the same. By scrapehero.com.
Tags python apis devops
-
Ray on databricks
Posted on January 31, 2022, Level intermediate Resource Length long
Ray is an open-source project first developed at RISELab that makes it simple to scale any compute-intensive Python workload. With a rich set of libraries and integrations built on a flexible distributed execution framework, Ray brings new use cases and simplifies the development of custom distributed Python functions that would normally be complicated to create. By Stephen Offer.
Tags data-science python machine-learning big-data
-
How Airbnb enables consistent data consumption at scale
Posted on December 25, 2021, Level intermediate Resource Length long
In this post, we will focus our story on how Minerva drastically simplifies and improves the data consumption experience for our users. Specifically, we will showcase how a unified metric layer, which we call the Minerva API, helps us build versatile data consumption experiences tailored to users with a wide range of backgrounds and varying levels of data expertise.
Tags data-science database cloud python distributed