Data Stream Sampling: Basic Techniques and Results | CS 410, Study notes of Computer Science

Download Data Stream Sampling: Basic Techniques and Results | CS 410 and more Study notes Computer Science in PDF only on Docsity!

11/7/

Data Streams: Lecture 13

CS 410/510 Data Streams Lecture 13: Data-Stream Sampling: Basic Techniques and Results

Kristin Tufte, David Maier

11/7/

Data Streams: Lecture 13

Data Stream Sampling „

Sampling provides a synopsis of a datastream

Sample can serve as input for ‰

Answering queries

‰

“statistical inference about the contents of thestream”

‰

“variety of analytical procedures”

Focus on: obtaining a sample from thewindow (sample size « window size)

11/7/

Data Streams: Lecture 13

Simple Random Sampling (SRS) „

What is a “representative” sample?

SRS for a sample of k elements from awindow with n elements ‰

Every possible sample (of size k) is equallylikely, that is has probability: 1/

‰

Every element is equally likely to be in sample

Stratified Sampling ‰

Divide window into disjoint segments (strata)

‰

SRS over each stratum

‰

Advantageous when stream elements closetogether in stream have similar values

n k

( )

11/7/

Data Streams: Lecture 13

Bernoulli Sampling „

Includes each element in the sample withprobability q

The sample size is not fixed, sample size isbinomially distributed

Probability that sample contains kelements is:

Expected sample size is nq

( )

q

k

(1-q)

n-k

n k

11/7/

Data Streams: Lecture 13

Binomial Distribution - Example

Binomial Distribution (n=20, q=1/3)

0

0.080.060.040.

0.180.160.140.

0

2

4

6

8

10

12

14

16

18

20

Probability

Sample Size

Expected Sample Size = 20*1/

11/7/

Data Streams: Lecture 13

8

Bernoulli Sampling - Implementation

Naïve: ‰

Elements inserted with probability q (ignored withprobability 1-q)

‰

Use a sequence of pseudorandom numbers (U

1

, U

2

U

3

, …) U

i

[0,1]

‰

Element e

i

is included if U

i

q

e

1

e

2

e

6

e

5

e

4

e

3

U

1

U

2

=0.1 U

3

U

4

U

5

U

6

e

7

U

7

Example q = 0.

Sample:

e

2

e

5

e

7

11/7/

Data Streams: Lecture 13

Geometric Distribution - Example

Geometric Distribution

q = 0.

0

0

2

4

6

8

10

12

14

16

18

20

Probability

Number of Skips (

i

11/7/

Data Streams: Lecture 13

Bernoulli Sampling - Algorithm

11/7/

Data Streams: Lecture 13

Reservoir Sampling „

Produces a SRS of size k from a window oflength n (k is specified)

Initialize a “reservoir” using first kelements

For every following element, insert withprobability p

i

(ignore with probability 1-p

i

p

i

= k/i for i>k

(p

i

= 1 for i

k)

‰

p

i

changes as i increases

Remove one element from reservoir beforeinsertion

11/7/

Data Streams: Lecture 13

Reservoir Sampling

Sample size 3

(k=3)

Recall: p

i

= 1 i

k, p

i

= i/k i>k

e

1

e

2

e

5

e

4

e

3

p

3

p

4

p

5

e

6

e

7

p

7

e

8

p

8

p

1

p

2

p

6

U

6

U

4

U

4

U

5

U

5

Reservoir Sample:

e

1

e

2

e

3

e

4

e

5

e

8

11/7/

Data Streams: Lecture 13

Reservoir Sampling - Observations „

Insertion probability (p

i

= k/i i>k)

decreases as i increases

Also, opportunities for an element in thesample to be removed from the sampledecrease as i increases

These trends offset each other

Probability of being in final sample is samefor all elements in the window

11/7/

Data Streams: Lecture 13

Other Sampling Schemes „

Stratified Sampling ‰

Divide window into strata, SRS in each stratum

Deterministic & Semi-DeterministicSchemes ‰

i.e. Sample every 10

th

element

Biased Sampling Schemes ‰

Bias sample towards recently-receivedelements

‰

Biased Reservoir Sampling

‰

Biased Sampling by Halving

11/7/

Data Streams: Lecture 13

Stratified Sampling „

When elements close to each other inwindow have similar values, algorithms suchas reservoir sampling can have bad luck

Alternative: divide window into strata anddo SRS in each strata

If you know there is a correlation betweendata values (i.e. timestamp) and position instream, you may wish to use stratifiedsampling

11/7/

Data Streams: Lecture 13

Deterministic Semi-deterministicSchemes „

Produce sample of size k by inserting everyn/k th element into the sample

Simple, but not random ‰

Can’t make statistical conclusions about windowfrom sample

Bad if data is periodic

Can be good if data exhibits a trend ‰

Ensures sampled elements are spreadthroughout the window

e

1

e

2

e

3

e

4

e

5

e

6

e

7

e

8

e

9

e

10

e

11

e

12

e

13

e

14

e

15

e

16

e

17

e

18

n=18, k=