Introduction About Site Map

XML
RSS 2 Feed RSS 2 Feed
Navigation

Main Page | Blog Index

Friday, February 22nd, 2013, 12:00 am

Splitting SQL Loads to Multiple CPU Cores

Foday I wrote the following bash script (note: written for 16 cores although it can be modified for more or for less), which splits SQL operations, preceded by “ [NUMERIC WEIGHT] |“. This helps assure a nice distribution of load on the bare metal. This might be reusable, so I decided to post it on my site.


IFS=$'\n'

CORE0LOAD=0
CORE1LOAD=0
CORE2LOAD=0
CORE3LOAD=0
CORE4LOAD=0
CORE5LOAD=0
CORE6LOAD=0
CORE7LOAD=0
CORE8LOAD=0
CORE9LOAD=0
CORE10LOAD=0
CORE11LOAD=0
CORE12LOAD=0
CORE13LOAD=0
CORE14LOAD=0
CORE15LOAD=0

WEIGHT=0;

echo Starting...

for line in $(cat ~/Desktop/input)
# while read line < ~/Desktop/input
# for line in `cat ~/Desktop/input`
do
# echo -e line="$line"
# $line="$line"
# echo $line
# exit
	if [ $CORE0LOAD -lt $CORE15LOAD ]
	then
# echo $line
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE0LOAD+$SCORE)) > total
		read SCORETOTAL < total
# echo $SCORETOTAL
		CORE0LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line >> core0.sh



	elif [ $CORE1LOAD -lt $CORE2LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE1LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE1LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core1.sh
	elif [ $CORE2LOAD -lt $CORE3LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE2LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE2LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core2.sh
	elif [ $CORE3LOAD -lt $CORE4LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE3LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE3LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core3.sh
	elif [ $CORE4LOAD -lt $CORE5LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE4LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE4LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core4.sh
	elif [ $CORE5LOAD -lt $CORE6LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE5LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE5LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core5.sh
	elif [ $CORE6LOAD -lt $CORE7LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE6LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE6LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core6.sh
	elif [ $CORE7LOAD -lt $CORE8LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE7LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE7LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core7.sh
	elif [ $CORE8LOAD -lt $CORE9LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE8LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE8LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core8.sh
	elif [ $CORE9LOAD -lt $CORE10LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE9LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE9LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core9.sh
	elif [ $CORE10LOAD -lt $CORE11LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE10LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE10LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core10.sh
	elif [ $CORE11LOAD -lt $CORE12LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE11LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE11LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core11.sh
	elif [ $CORE12LOAD -lt $CORE13LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE12LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE12LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core12.sh
	elif [ $CORE13LOAD -lt $CORE14LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE13LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE13LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core13.sh
	elif [ $CORE14LOAD -lt $CORE15LOAD ]
	then
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE14LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE14LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line  >> core14.sh
	else
# echo $line
		echo $line > current_line
		cut -b 1-13 current_line > current_number
		read SCORE < current_number
		echo $(($CORE15LOAD+$SCORE)) > total
		read SCORETOTAL < total
		CORE15LOAD=$((SCORETOTAL))
		cut -b 16-10000 current_line >> core15.sh
	fi
done
echo Ended...
 

Now, each file contains operations to be run on one core, without conflict (order or load imbalance).

Friday, February 15th, 2013, 12:15 pm

BT Connection Throttling (Or How BT Screws Its Customers)

FOR ALMOST a month now BT has been throttling speeds on its exchange, thus crippling/impeding connections, breaking what was previously working without issues. Interlacing and full capacity, they appear to have decided, can just arbitrary be disabled, disconnecting people with all their lives sessions at any time. When phoning BT — as I have done for about 3 hours over the past month (in total) — they acknowledge the issue, although their technical staff seems unable to even recognise the problem or communicate it properly (among peers), which leaves them having to issue pathetic compensation (also requires hours on the line) while continuing to throttle the connections. This is a technical issue at their end — one that they can address and resolve remotely, but why do they knowingly break their systems in the first place? They make a lot of verbal promises — hence not legally binding — only to break those promises and leave people like me wasting a dozen or so hours, incapable of doing proper work or even keep good temper.

BT are failing because their own traffic management systems are defective by design. Calling this inevitable error is like saying that accidents are OK because you feel like committing traffic violations. This was not the case in prior years. Why is BT crippling its own infrastructure at cost to the clients? Why are they abusing their monopoly on landlines? Maybe it’s time to really break this monopoly rather than hide it behind the fake impression of “choice”. They know that people cannot cease reliance on landline (bar mobile networks), so they continue to throttle connections to the point where people get none of what they paid for. They just get a sporadic connection and a headache.

Saturday, February 9th, 2013, 11:14 am

Juice Media on Privacy

Thursday, February 7th, 2013, 12:20 am

Old Photo in the GIMP

IN GIMP there are numerous scripts or macros, or even effects and filters as they are more commonly called. One of them, a nice mixture of filters, is called “Old Photo” and it generates something like this gallery that I published some minutes ago.

GIMP is truly fantastic software and my wife intends to learn how to use it. I have made a huge number of images with the GIMP, maybe shy of 10,000 (since 2001).

Friday, February 1st, 2013, 3:02 pm

Benchpress After a Decade

I have not practiced this for years. It was never my strong point, but a decade ago when I had more muscle mass I could benchpress about 175 KG. This video of me at at 88 KG shows me barely doing 150 KG (I will try again Sunday, going nearer to the chest). I also filmed Kieran doing 100 KG and 97.5 after initially failing at 100 KG and then Jordan at 112.5 KG.

Kieran fails to lift 100.0 KG

Kieran lifts 97.5 KG

Kieran lifts 100.0 KG

Jordan lifts 100.0 KG

Jordan lifts 112.5 KG

Roy lifts 150.0 KG

Wednesday, January 23rd, 2013, 9:25 pm

TNT Sucks

TNT Sucks

TNT is not so well known as a courier. Maybe it’s better that way. Nobody should ever use TNT as a courier. Their service is poor and they charge a lot for this very poor service.

Context of our 4-hour experience with TNT is as follows. After my wife had spent an entire morning trying to get TNT to get their collective behinds together I fired off the following polite message (at lunchtime):

I am writing on behalf of my wife, who wasted about 4 hours today trying to get your service to import a document, all in vain (the service was very unsatisfactory). When she used the E-mail address you gave her it bounced back with an error (see below). So I am attempting to send it from my own account. Please phone her to acknowledge receipt of this message with this form.

This is an URGENT MATTER.

Regards,

Dr. Roy S. Schestowitz

Attached was the completed form, already nicely scanned and ready for action from TNT (where immediate service should be critical given their target customers).

Here is my wife’s original message:

To whom it may concern

I am not a client of TNT yet. Here is the information needed for me to do any transaction with the above courier.

No response all day. In fact, it bounced although it was sent to the correct address. The error received was as follows:

Reporting-MTA: dns;GBAHES628.ics.express.tnt
…
Action: failed
Status: 5.1.1
Diagnostic-Code: X-Notes;… not
listed in Domino Directory

After wasting an entire day losing valuable time and suffering a bit of fury and anger (directed only at TNT) we decided to throw TNT where it belongs, or rather, throw some TNT at TNT:

My wife and I would like to cancel this transacting. Your service has been utterly appalling and the excuses very poor. Your staff is unable to communicate internally, so the explanation needed to be provided 5 times. The E-mail to your address, which you provided repeatedly, bounced back to the sender with the following message from IBM Domino (it had been sent from Yahoo!, which is no ISP/relay that should be blacklisted).

So, after forms and details had been prepared for hours this proved to be an exercise in futility. 5 times in recent months we used DHL and never have we had to jumped through such hoops and receive deficient service. TNT said they would call when the order is received, but no contact was made. In fact, rather than the customer coming first it seemed assured that we were way down the list because we are no Big Business which makes up a large source of revenue for TNT. Keeping the customers waiting in vain is worse than doing nothing at all. We will never do business with TNT again and we shall discourage others who ponder doing so.

Regards,

Dr. Roy S. Schestowitz

If this post harms TNT’s reputation, then we are happy, not sorry. TNT deserves this.

Sunday, January 20th, 2013, 5:00 pm

Fullstack Circuit

Age 31, 89 KG, after a one-hour workout session, all natural (no supplements). This video was taken just a couple of hours ago despite the fact I had done a heavy workout beforehand. In the future I will do more repetitions, not just 4-5. In my early 20s I could do better, but on the shoulder press I struggled a little at maximum weight.

Real-time Posts

Posts by @schestowitz

Retrieval statistics: 23 queries taking a total of 0.097 seconds • Please report low bandwidth using the feedback form
Original styles created by Ian Main (all acknowledgements) • PHP scripts and styles later modified by Roy Schestowitz • Help yourself to a GPL'd copy
|— Proudly powered by W o r d P r e s s — based on a heavily-hacked version 1.2.1 (Mingus) installation —|