mingbo's tech tips

Graphite will align timestamp based on the retention defined in storage-schemas.conf,  whenever you send a metric to Graphite, Graphite will align timestamp like this: timestamp  - (timestamp % interval) so for example, let's say you have 1 minute interval (time per point), or  60  seconds for one of your metrics, then you send: my.useful.metric 2022 1414516239 Graphite will save (timestamp aligned): my.useful.metric 2022 1414516200 because 1414516239 - 1414516239 % 60 = 1414516200 and if you send multiple values with same timestamp, or you send multiple values then all of them  are within defined interval ----- that is: (timestamp  - (timestamp % interval)) has same value. the last one wins . Graphite will only save last value ( with exceptions, see detail in last part of this post ). So back to previous example, if you send in below order (doesn't matter when you actually send them, with  exceptions ): my.useful.metric 322   1414516229  my.useful.metric 52

It's difficult to find out who's using the security group if you looked at wrong place. Here's how to find out dependent objects: 1. Search group name or group ID at "Network Interfaces" under EC2 Management Console. This will give you all direct usage from instances including EC2, RDS, RedShift .... 2. find out dependencies between/among security groups by using this small program:   https://github.com/mingbowan/sgdeps you will get something like: python sgdeps.py --region us-east-1 mingbotest-A sg-b4566ad1 (mingbotest-A) |-- sg-9b566afe (mingbotest-C2) |-- sg-9f566afa (mingbotest-C1) |  |-- sg-69576b0c (mingbotest-D2) |  |  `-- sg-b4566ad1 (mingbotest-A) ** loop |  `-- sg-64576b01 (mingbotest-D1) |     `-- sg-9b566afe (mingbotest-C2) |-- sg-8b566aee (mingbotest-B2) `-- sg-86566ae3 (mingbotest-B1) then delete dependent rules/groups.

the solution I got runs more than 1 second, which is terrible. Please help if you have better algorithm """ Project Euler Problem #14 ========================== The following iterative sequence is defined for the set of positive integers: n->n/2 (n is even) n->3n + 1 (n is odd) Using the rule above and starting with 13, we generate the following sequence:                   13->40->20->10->5->16->8->4->2->1 It can be seen that this sequence (starting at 13 and finishing at 1) contains 10 terms. Although it has not been proved yet (Collatz Problem), it is thought that all starting numbers finish at 1. Which starting number, under one million, produces the longest chain? NOTE: Once the chain starts the terms are allowed to go above one million. """ limit=1000000 uplimit  = limit * 3 answer = 1 maxval = 0 data = [0] * uplimit data[1] = 1 for ind in xrange(1,limit

""" Project Euler Problem #12 ========================== The sequence of triangle numbers is generated by adding the natural numbers. So the 7^th triangle number would be 1 + 2 + 3 + 4 + 5 + 6 + 7 = 28. The first ten terms would be:                  1, 3, 6, 10, 15, 21, 28, 36, 45, 55, ... Let us list the factors of the first seven triangle numbers:    1: 1    3: 1,3    6: 1,2,3,6   10: 1,2,5,10   15: 1,3,5,15   21: 1,3,7,21   28: 1,2,4,7,14,28 We can see that 28 is the first triangle number to have over five divisors. What is the value of the first triangle number to have over five hundred divisors? """ limit = 100000 data=[0] * limit for i in xrange(1,int(limit/2+1)):     data[i::i] = [ x+1 for x in data[i::i]] def find():     s = 1     while True:         if s % 2 == 0:             total = data[s//2] * data[s+1]         else:             total = data[s]*data[(s+1)//2]         if total > 500:             print s

basically, when any process need to do I/O and underlying library calls io_schedule() will put CPU into iowait status and then if the CPU is idle (no other active process running), iowait time increases. below is related code snip from 3.x kernel: // in kernel/sched/cputime.c                                                                                                                                                                   /*  * Account for idle time.  * @cputime: the cpu time spent in idle wait  */ void account_idle_time(cputime_t cputime) {     u64 *cpustat = kcpustat_this_cpu->cpustat;     struct rq *rq = this_rq();                                      // get running queue of this processor/CPU     if (atomic_read(&rq->nr_iowait) > 0)                 // if there's process in iowait status         cpustat[CPUTIME_IOWAIT] += (__force u64) cputime;     else         cpustat[CPUTIME_IDLE] += (__force u64) cputime; } // in kernel/sched/cor

I searched online for answers, but none of them are good enough for me --- most of them says it's equal to running queue, but some observations on an I/O busy NFS server disagree with that. So I checked source code. TLDR;  load average can be considered as a sum of total running processes (the run queue size) and interruptible process over specific time period. This applies to at least version 2.6.x and 3.x (I only checked source code of those two versions) BTW, the uptime command is only read and output the data in /proc/loadavg for load average. (A little bit) Long but official explanation : copied from Linux kernel source code: of v3.14-rc6 /* * Global load-average calculations * * We take a distributed and async approach to calculating the global load-avg * in order to minimize overhead. * * The global load average is an exponentially decaying average of nr_running + * nr_uninterruptible. * * Once every LOAD_FREQ: * * nr_active = 0; * f