MongoDB documents of calulated values for a dashboard vs re-retrieving on each web page view? -


if have page in web app displays dashboard type statistics documents in database (counts, docs created per hour, per day etc), best pre-calculate data , store in separate document (and update needed), or assuming collections have appropriate indexes, appropriate execute queries retrieve these statistics on every load of page?

it's not necessary data has date on every page hit/load, that's why thinking maintain data need display in separate document can retrieved on page hit (or cached , re-retrieved every 5 minutes or similar).

that's pretty broad, , have feeling have identified key points. speaking, should consider these questions:

  • do need allow users apply filters? complex filters make pre-aggregation impossible.
  • related: exact same data ever queried again? if not, pre-aggregation might need happen on different levels of granularity (e.g. creating day / week / month totals , summing these, instead of individual events).
  • what relation of reads vs. writes on data? if number of writes small, might ok keep counters in real-time, instead of using read-caching.
  • what performance requirements cached , uncached queries? getting fast cached queries trivial, comes @ cost of stale data. making uncached queries faster more tricky , requires multi-level approach discussed before - doesn't if old data comes super fast, new queries take minutes.

caching works if data can't changed later (or seldomly changed), , queries remain same chance of re-occuring. nice example facebook's profiles, past years apparently cached every visitor-profile combination. first accesses slow, however...


Comments

Post a Comment

Popular posts from this blog

cakephp - simple blog with croogo -

i not problem, made simple blog in croogo, problem is not adding or modifying or delete, because arabic language ?? function add in controller `public function admin_add() { $this->set('title_for_layout', __('add part')); if (!empty($this->request->data)) { $this->part->create(); if ($this->part->save($this->request->data)) { $this->session->setflash(__('the part has been saved'), 'default', array('class' => 'success')); $this->redirect(array('action' => 'index'));//, $this->part->id)); } else { $this->session->setflash(__('the part not saved. please, try again.'), 'default', array('class' => 'error')); } } }` => model `class part extends appmodel { public $name = 'part'; var $belongst...
Read more

How to group boxplot outliers in gnuplot -

i have large set of data points. try plot them boxplot, of outliers exact same value , represented on line beside each other. found how set horizontal distance between outliers in gnuplot boxplot , doesn't much, apparently not possible. is possible group outliers together, print 1 point , print number in brackets beside indicate how many points there are? think make more readable in graph. for information, have 3 boxplots 1 x value , times 6 in 1 graph. using gnuplot 5 , played around pointsize, doesn't reduce distance anymore. hope can help! edit: set terminal pdf set output 'dat.pdf' file0 = 'dat1.dat' file1 = 'dat2.dat' file2 = 'dat3.dat' set pointsize 0.2 set notitle set xlabel 'x' set ylabel 'y' header = system('head -1 '.file0); n = words(header) set xtics ('' 1) set [i=1:n] xtics add (word(header, i) i) set style data boxplot plot file0 using (1-0.25):1:(0.2) boxplot lw 2 lc rgb '#8b0000...
Read more

bash - Performing variable substitution in a string -

i have string contains use of variable, , i'd substitute variable's value string. right best have is #!/bin/bash foo='$name; echo bar' name="the name" expanded="$(eval echo "$foo")" echo "$expanded" which has obvious defects: prints the name bar while i'd print the name; echo bar instead of eval can bash's regex matching bash's string replacement: foo='$name; echo bar' name="the name" [[ "$foo" =~ \$([[:alnum:]]+) ]] && s="${!bash_rematch[1]}" && expanded="${foo/\$${bash_rematch[1]}/$s}" echo "$expanded" name; echo bar
Read more