Python Freelancers

I have a very large dataframe (about 1.1M rows) and I am trying to sample it. I have a list of inde… Read more Quickly Sampling Large Number Of Rows From Large Dataframes In Python

I'm trying to multiplicate 2 big matrices with memory limit using hdf5 (pytables) but function … Read more Matrix Multiplication Using Hdf5

I have a 10 billion line tab-delimited file that I want to split into 5,000 sub-files, based on a c… Read more Split 10 Billion Line File Into 5,000 Files By Column Value In Perl Or Python

So, I'm running this simple program on a 16 core multicore system. I run it by issuing the foll… Read more Why Is This Simple Spark Program Not Utlizing Multiple Cores?

I have a large data-set (I can't fit entire data on memory). I want to fit a GMM on this data s… Read more Sklearn-gmm On Large Datasets

I am working with Python on Spark and reading my dataset from a .csv file whose first a few rows ar… Read more Python (pyspark) Error = Valueerror: Could Not Convert String To Float: "17"

I'm working on a big data project for my school project. My dataset looks like this: https://gi… Read more How To Predict Correctly In Sklearn Randomforestregressor?

I am creating a co-occurring matrix, which is of size 1M by 1M integer numbers. After the matrix i… Read more How To Incrementally Create An Sparse Matrix On Python?

I have a large data-set (I can't fit entire data on memory). I want to fit a GMM on this data s… Read more Sklearn-GMM On Large Datasets

I have a dataframe with a rough structure like the following: +-------------------------+----------… Read more PySpark: Inconsistency In Converting Timestamp To Integer In Dataframe

Let us Assume, I have a key value pair in Spark, such as the following. [ (Key1, Value1), (Key1, Va… Read more How Can A Reduce A Key Value Pair To Key And List Of Values?