Traditional sampling techniques (grid vs random vs sobol vs latin hypercube)
Vložit
- čas přidán 1. 07. 2024
- Welcome to video #1 of the Adaptive Experimentation series, presented by graduate student Sterling Baird @sterling-baird at the 18th IEEE Conference on eScience in Salt Lake City, UT (Oct 10-14, 2022). In this video, Sterling introduces the concept of adaptive experimentation and covers traditional sampling approaches, including grid, random, Latin hypercube, and Sobol sampling. He also discusses the use of discrepancy as a metric for performance evaluation. Don't miss the next installment in this informative series on experimental optimization.
Github link to jupyter notebook github.com/sparks-baird/self-...
next video in series: • Traditional sampling t...
0:00 introduction to adaptive experimentation
2:09 Comparing grid/random search with quasi-random search with adaptive experimentation approaches (grid vs human intuition)
4:40 traditional optimization jupyter notebook tutorial
6:14 grid sampling
7:45 latin hypercube and sobol sampling
9:36 comparing different sampling
11:05 discrepancy comparison in low and high dimensional data - Hudba
Awesome series! Thank you so much for this!
One thing that came to mind while watching this video:
The quasi-random methods presented assume a continuous space of independent variables.
If you have a discrete set of materials (e.g., a set of recipes created according to a particular subject logic), these assumptions could be violated. This could be the case if the amount of one ingredient is in a certain ratio to another, the materials have different densities, and the materials are described by a finite volume.
Here, random selection has the advantage of not making any assumptions. To minimize the discrepancy, one could use k-means.
This is a really great comment. We actually have a follow-up video a few down where you'll see we implement a constrained search space
@@TaylorSparks Great - that is very helpful! I think the constraints can narrow down the continuous space. My comment referred specifically to a discrete space, where Sabol, Latin Hyperc Cube, etc. will not work in principle if there are dependencies between variables. What is meant here is the case where one has, for example, a given list of recipes or a data matrix.
Wonder how well sampling based on space-filling curves (like Hilbert’s) could perform. Great discussion, have always been interested in such measures (like discrepancy). 🤗
Took me a bit of digging, but it looks like sobol beats it books.google.co.uk/books?id=S9ynGgjgs1sC&pg=PA476&lpg=PA476&dq=hilberts+space+filling+curve+sampling+vs+sobol&source=bl&ots=MMsY3Fz1y3&sig=ACfU3U0Koa7w0X2JYre_WiTGX7azLwJEcQ&hl=en&sa=X&ved=2ahUKEwiw7Z79-4z8AhWIUcAKHfLEDAo4ChDoAXoECAMQAg
@@TaylorSparks That’s awesome. I’ll dig in too, interesting problem for sure. Thanks for the link. 🤗
See the lecture, “a unified approach to discrepancy minimization” by nikhil bansal here on CZcams. Gives a good description of discrepancy minimization. 🤗
I was looking for a comparison between Sobol and BRJ (Basic Random Jump). I did not know about Latin Hypercube. Do you have any insight on BRJ, ever looked into it? Cheers
We haven't looked into it. If you come across a comparison, we'd be interested to hear!
Hi Taylor, great video! Can you provide the figure of minute 12:40? the jupiter notebook doesnt work for me :/
Sure, send me an email and I will reply with the image
But what's the point in not evenly distributing your data points? If you're searching for effects then shouldn't you be doing that?
It's counterintuitive right? You can find trends more efficiently through random data points and even more efficiently through quasi random and best of all through Bayesian statistics.