![]() ![]() reduce ( lambda x, y : ( x + y, x + y ) ) x_bar_sketch = sketch_mean / float ( sketch_mean ) print ( "Mean via Sketching:" ) x_bar_sketch Mean via Sketching : 50.128590000000003 Example: VarianceĮxample: Variance + Partition and Model def var_part ( interator ): x = 0 x2 = 0 n = 0 for i in interator : x += i x2 += i ** 2 n += 1 avg = x / float ( n ) var = ( x2 + n * avg ** 2 ) / ( n - 1 ) return ] var_part_model = X_part. def ( theta ): total = 0 weighted_avg = 0 for i in xrange ( len ( theta )): weighted_avg += theta * theta total += theta theta_bar = weighted_avg / total return theta_bar print ( "Mean via Partition and Model:" ) weighted_avg ( model_mean_part ) Mean via Partition and Model : 50.128590000000003 Example: Mean + Sufficient Statistics / Sketching sketch_mean = X_part. The partitions are not equally sized, thus we’ll use a weighted average. collect () model_mean_part, ,, ,, ,, ,, ] Example: Mean + Partition and Model X = sc.parallelize(randint(0, 101, 200000))Įxample: Mean + Partition and Model def mean_part ( interator ): x = 0 n = 0 for i in interator : x += i n += 1 avg = x / float ( n ) return ] model_mean_part = X_part.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |