Added sample_shape parameter to RandomVariable

[matthewdhoffman]

This is a small PR that adds a keyword argument sample_shape to Edward RandomVariables that gets passed to the underlying tf.contrib.distributions Distribution's sample function. This lets us replace syntax like ed.Normal(mu=tf.ones([10, 1])*mu_vec, sigma=1.) with ed.Normal(mu=mu_vec, sigma=1., n=10).

IMO, this is clearer, since it explicitly says that some slices are i.i.d., some are independent, and dependent. (Distribution keeps track of these with batch_size and event_size.) It's also (very marginally) more efficient, and it makes doing algebra on the graph a little easier in some cases.

[dustinvtran]

Cool! I'll push commits to this now.

An example where sample_shape is not only more efficient but necessary is with the Dirichlet process. DirichletProcess(tf.ones(N), Normal(0.0, 1.0)) represents a batch of N independent Dirichlet processes (each with concentration parameter 1.0 and std normal as base measure); this means no atoms are shared across their draws. DirichletProcess(1.0, Normal(0.0, 1.0), sample_shape=N) represents N samples from the same Dirichlet process; this is the correct generative process for mixture models and etcetera.

[dustinvtran]

I checked if there were any consequences during inference. For example, the user might write

from edward.models import Normal

z = Normal(tf.zeros(100), tf.ones(100))
qz = Normal(tf.Variable(0.0), tf.nn.softplus(tf.Variable(0.0)), sample_shape=100)

inference = ed.KLqp({z: qz})

which is weird. Fortunately the broadcasting during log-density calculations still work, so all variational inference methods are fine (including MAP, even though PointMass with sample_shape is really weird).

I made MonteCarlo raise an error if its Empirical approixmations use a non-scalar sample_shape. Things might go awry especially for auxiliary methods which use momentum.

[dustinvtran]

@matthewdhoffman | This is ready for you to take a look at again. If you approve, feel free to merge (using the "Squash and merge" option); comments welcome.

[matthewdhoffman]

That KLqp example is very interesting. It's something that someone might conceivably want to do (e.g., maybe one could use it to share variances but not means across a bunch of examples). Hopefully it doesn't trip anyone up.

LGTM, merging.