but I encountered rrd file writing timeout(12 seconds) as the number of rrd files increased.
So I started investigating where the performance bottleneck could be occurring ( We have multi-core CPU and there are no cpu or I/O bottlenecks ) and parallelized the time-consuming job in insert.pl like this with Parallel::Forker module.
Performance increases almost linearly with increasing N until you meet the limit of I/O bandwidth.
Original
JOB FLOW
perfdata in array -> iteration ( parse, write ) [ @perfdata ] [ processdata(@perfdata) ]
CODE
processdata(@perfdata); debug( 5, 'INSERT nagiosgraph exited' );
Parallelized
JOB FLOW
perfdata in array -> split array into N -+-> array 0 -> iteration ( parse, write )
+-> array 1 -> iteration ( parse, write )
+-> . -> iteration ( parse, write )
+-> array N-1-> iteration ( parse, write ) CODE
use Parallel::Forker;
my $fork = Parallel::Forker->new;
$SIG{CHLD} = sub { Parallel::Forker::sig_child($fork); };
$SIG{TERM} = sub { $fork->kill_tree_all('TERM') if $fork && $fork->in_parent; die "Quitting...\n"; };
my $n = 4;
my @n_perfdata;
my $size = int(@perfdata/$n);
foreach ( 0 .. $n-1 ){
$n_perfdata[$_] = [ splice @perfdata, 0, $size ];
}
push @{$n_perfdata[$n-1]}, @perfdata;
foreach ( 0..$#n_perfdata ) {
$fork->schedule ( name => $_, run_on_start => \&child_sub )->run;
}
$fork->wait_all();
debug( 5, 'INSERT nagiosgraph exited' );
sub child_sub {
processdata( @{ $n_perfdata[ $_[0]{name} ] } );
}
New Opsview 3.9.1 improved the performance by other method.
답글삭제This is considered obsolete now.