but I encountered rrd file writing timeout(12 seconds) as the number of rrd files increased.
So I started investigating where the performance bottleneck could be occurring ( We have multi-core CPU and there are no cpu or I/O bottlenecks ) and parallelized the time-consuming job in insert.pl like this with Parallel::Forker module.
Performance increases almost linearly with increasing N until you meet the limit of I/O bandwidth.
Original
JOB FLOW
perfdata in array -> iteration ( parse, write ) [ @perfdata ] [ processdata(@perfdata) ]
CODE
processdata(@perfdata); debug( 5, 'INSERT nagiosgraph exited' );
Parallelized
JOB FLOW
perfdata in array -> split array into N -+-> array 0 -> iteration ( parse, write ) +-> array 1 -> iteration ( parse, write ) +-> . -> iteration ( parse, write ) +-> array N-1-> iteration ( parse, write )
CODE
use Parallel::Forker; my $fork = Parallel::Forker->new; $SIG{CHLD} = sub { Parallel::Forker::sig_child($fork); }; $SIG{TERM} = sub { $fork->kill_tree_all('TERM') if $fork && $fork->in_parent; die "Quitting...\n"; }; my $n = 4; my @n_perfdata; my $size = int(@perfdata/$n); foreach ( 0 .. $n-1 ){ $n_perfdata[$_] = [ splice @perfdata, 0, $size ]; } push @{$n_perfdata[$n-1]}, @perfdata; foreach ( 0..$#n_perfdata ) { $fork->schedule ( name => $_, run_on_start => \&child_sub )->run; } $fork->wait_all(); debug( 5, 'INSERT nagiosgraph exited' ); sub child_sub { processdata( @{ $n_perfdata[ $_[0]{name} ] } ); }
New Opsview 3.9.1 improved the performance by other method.
답글삭제This is considered obsolete now.