Michael Berk – Medium

Michael Berk

Michael Berk
in
Towards Data Science

1.5 Years of Spark Knowledge in 8 Tips

My learnings from Databricks customer engagements

Dec 24, 2023

spark partition data skew optimize optimization pyspark sql python UI partition

Dec 24, 2023

Michael Berk
in
Towards Data Science

HyperOpt Demystified

How to automate model tuning with HyperOpt

Nov 8, 2022

Hyperparameter hyper parameter tuning model tuning machine learning data science sklearn model mllib spark hyperopt tree parzen estimator tpe tree based parzen esimtator mlflow databricks

Nov 8, 2022

Michael Berk
in
Dev Genius

How to Automate Your Data Infrastructure with Code

What is Terraform and why should you use it

Sep 8, 2022

Terraform Infrastructure as code iac automation aws Amazon web services clouds computing azure data engineering what is terraform tutorial how to use best

Sep 8, 2022

Michael Berk
in
Towards Data Science

Demystifying the Parquet File Format

The default file format for any data science workflow

Aug 16, 2022

Demystifying the Parquet File Format

Aug 16, 2022

Michael Berk
in
Towards Data Science

PySpark Data Skew in 5 Minutes

Exactly what you need, and no more

May 10, 2022

PySpark Data Skew in 5 Minutes

May 10, 2022

Michael Berk
in
Towards Data Science

SQL to PySpark

A quick guide for moving from SQL to PySpark.

May 6, 2022

SQL to PySpark

May 6, 2022

Michael Berk
in
Towards Data Science

How does linear regression really work?

The math and intuition behind ordinary least squares (OLS)

Feb 23, 2022

How does linear regression really work?

Feb 23, 2022

Michael Berk
in
Towards Data Science

5 Advanced Tips on Python Objects

Python is an object oriented programming language but can behave strangely. If you come from other OOP languages, this post may benefit you

Feb 9, 2022

5 Advanced Tips on Python Objects

Feb 9, 2022

Michael Berk
in
Towards Data Science

Don’t Use a T-Test for A/B Testing

How to use multiple linear regression to determine ATE and statistical significance

Feb 2, 2022

Don’t Use a T-Test for A/B Testing

Feb 2, 2022

Michael Berk
in
Towards Data Science

5 Advanced Tips on Python Decorators

Do you want to write concise, readable, and efficient code? Well, python decorators may help you on your journey.

Jan 24, 2022

5 Advanced Tips on Python Decorators

Jan 24, 2022

Michael Berk

Michael Berk

I’m a Data Scientist writing 52 posts that bring academic research to DS industry. https://www.linkedin.com/in/michael-berk-48783a146/

Following

Help
Status
About
Careers
Press
Blog
Privacy
Terms
Text to speech
Teams