Thursday, December 17, 2009

Abstraction


Source: http://imgs.xkcd.com/comics/abstraction.png

Friday, December 11, 2009

Blizzard: modeling the climate change

Impressive and elegant new German supercomputer:



Specifications:

HPC Cluster ("blizzard")

  • IBM p575 "Power6" cluster
  • water cooled
  • 16 dual core CPUs per node
  • Total size: 264 nodes, 8448 cores
    • 249 compute nodes
    • 12 I/O nodes
    • 3 interactive nodes
  • 18,8 GigaFlops/core
  • Total system peak performance: 158 TeraFlops/s
  • Top500: Rank 27 in 06/09
  • 20 TeraByte memory
  • 3 PetaByte GPFS file system (additional 3 PetaByte in 2011)
  • 8 x Qlogic 288 port 4 x DDR Infiniband switches
    • bandwidth: 16 GigaByte/s node to node (bidirectional)
    • 25.4 km Infiniband cables
  • http://www.dkrz.de/dkrz/about/hardware/HPC_Cluster

But Blizzard is connected to the Data archive:
  • HPSS: High Performance Storage System
  • 6 Sun StorageTek SL8500 tape libraries
    • 8 robots per library
    • 10 000 media slots per library
  • 73 tape drives
  • total capacity: 60 PetaByte.
  • 60.000 media (T10000 A/B, LTO4, 9940B and 9840C )
  • 500 TeraByte disk cache
  • projected fill rate: 10 PetaByte/year
  • bidirectional bandwidth: 3 GigaByte/s (sustained), 5 GigaByte/s (peak)
  • http://www.dkrz.de/dkrz/about/hardware/DataArchive
and the visualization machine:
  • 8 Vis Nodes (sva91 ... sva98)
    • HP XW 9400 Workstation
    • 2 Dual Core Opteron64 2.6 GHz
    • 32 GB RAM
    • 2 NVidia FX 5500 Graphics with 1 GB RAM
    • InfiniBand HCA
  • 1 SMP Vis Node (sva90)
    • HP ProLiant DL585 Server
    • 4 Dual Core Opteron64 2.4 GHz
    • 128 GB RAM
    • 2 NVidia FX 5500 Graphics with 1 GB RAM (via NVidia QuadroPlex)
    • InfiniBand HCA
  • 1 System Management Node (sva99)
    • HP ProLiant DL 385 Server
    • 2 Dual Core Opteron 2.2 GHz
    • 4 GB RAM
    • InfiniBand HCA
  • HP SFS (Scalable File Share) 72 TByte brutto / 48 TByte netto
    • 2 Meta Data Server (MDS)
    • 6 Object Storage Targets (OST)
    • Infiniband HCAs
  • http://www.dkrz.de/dkrz/about/hardware/vis_cluster

Wednesday, December 9, 2009

Mac and HAL

An old commercial, Think different Dave ...

Monday, November 30, 2009

Wednesday, November 18, 2009

Computer graphics in Star Wars (1977)

This is a video documentary about how one scene in Episode IV was prepared, the scene in particular is when the rebel pilots were planning how to attack the Death Star. The Commander showed them an animation of the Death Star and where to fire.



Of course, this is 1976, no high end computer graphics cards exists yet, so Larry Cuba describe the steps in the 3D modeling based on photographs of the real miniature models. Check how he's controlling the rotation in real time (awesome) with big controls (no mouse neither exists).

The final touch was to render each scene, shot by shot, in the computer screen with and automatic camera capturing in film.

Via Topless Robot.

Monday, November 16, 2009

Updating nightmare

Before I must say, I have a laptop with dual boot, Mandriva in Dr. Jekyll and MS-Vista was Mr. Hyde. Yes, it was because two days ago I decided to update to the well recommended Win7. Few days ago I received a legal and nice DVD with the OS, the 64 bits version to upgrade Vista Home Premium. Then, I backup all my personal data.

My laptop is an Acer Aspire 5520, AMD Athlon 64 x2, 2 GB RAM, 160 GB HD, Nvidia 7000M, since I have it, I never touch the disc partitions, because the disc was already divided into 3, one part for the Acer recovery tools (3 GB), 70 GB for Vista, and the rest I used for Mandriva (from 2007 to now 2010 versions).

Everything works, well, of course Vista was slow and frustrating, I keep it because I share this computer and when I had time, I played games unsupported in Linux. After the bunnies convinced me to upgrade, I put the disc and upgrade to W7. Easy, after few clicks and 1 hr, W7 was installed. I rebooted the laptop, and waited for the final installation. I you are familiar with this installations you must knew that it takes time, so I was doing other stuff and just checking when it needs some clicks, that was my error, when the SO reboot, the Acer recovery mode entered and tried to restore Vista, blowing the whole hard disk!

Sadly, I tried to recover with a Mandriva One liveCD, everything was lost, but I was a backup for my data, no problem. So I modified the hard disk to remove the recovery partition and create a fresh installation for both SO. I tried to install W7, but my first mistake was not formatted the partition in NTFS mode, W7 said it cannot be installed in a FAT32 partition (wtf, just format it!), again, I use the liveCD, formatted this and installed W7. When it finished (another lost hour), it asked for the Product Key and cannot validate! I take a rest, watch a movie (Nicholas Cage's "Knowing") and came back with the solution, my disk was for a Vista UPGRADE, so I reinstalled Vista from my laptop original DVDs (another hour), then I use the W7 DVD and voila, it worked!

Finally I installed Mandriva 2009, upgraded to 2010, it was more simple than MS migration, and now both SO live together in my computer. I cannot believe people said that private SO are better than GNU SOs. BTW now I'm using KDE 4.3, let's see if it can convince me to leave Gnome.

What is my opinion with this new SOs?
  • W7 is better than Vista, of course if you only have worked with Vista you feel the difference, I'm not impressed because I know other SOs (Linux/Mac OS X). W7 is cute, little fast, and after one day using it, it works fine.
  • KDE/Mandriva 2010 is also pretty, faster and really usable.
Should I migrate Mac OS X Leopard to Snow Leopard now?

Monday, November 9, 2009

The history of computing (video)

The next is nice and educational video about computer history and the future, I like the analogies with biological systems.

Trillions from MAYAnMAYA on Vimeo.

Tuesday, October 27, 2009

Random lines in a text file

Sometimes I need to sample large data sets, so I randomly select some lines in the file. My files are generally text-records, one record by line, then I wrote this small script to do the task:

#!/usr/bin/perl -w
use strict;

=head1 NAME

selectRandomLines.pl

=head1 DESCRIPTION

Select random lines in a file.

=cut

$ARGV[2] or die "Usage: selectRandomLines.pl TOTAL_LINES_IN_FILE NUM_LINES_WANTED FILE_NAME\n";

my $total = shift @ARGV; # Total lines in the file
my $want = shift @ARGV; # Total lines to select
my $file = shift @ARGV; # The file
my $line = 0; # Line counter
my %select = randomSelect($total, $want); # Hash with selected lines

open FILE, "$file" or die "cannot open $file\n";
while (<FILE>) { print "$_" if (defined $select{$line++}); }
close FILE;

=head1 SUBROUTINES
randomSelect()
CALL: randomSelect(TOTAL_ELEMENTS, TOTAL_WANTED) [NUM, NUM]
RETURN: %s [HASH]
=cut
sub randomSelect {
my $t = shift @_;
my $n = shift @_;
my %s = ();
for (my $i = 0; $i < $n; $i++) {
my $v = int(rand $t);
(defined $s{$v}) ? $i-- : $s{$v}++;
}
return %s;
}

=head1 AUTHOR

JC @ 2009

=head1 LICENSE

This is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with code. If not, see .

=cut

Monday, October 12, 2009

Error in DBI::SQLite

I was coding to query a SQLite database using Perl::DBI, I have a small code like this:
#!/usr/bin/perl -w
use strict;
use DBI;
my $dbh = DBI->connect("dbi:SQLite:dbname=db_file", "", "") or die "error in connection\n";
my $sth = $dbh->prepare($sql) or die "cannot prepare SQL\n";
$sth->execute();
while (my @data = $sth->fetchrow_array()) {
#process data
}
$sth->finish;
$dbh->disconnect;
But, every time I execute this script I obtain this warning:
closing dbh with active statement handles .... blah, blah, blah
I double checked the code, the documentation and ran some debug examples, I obtained the expected result, so why this message? Finally, I found the solution in the PerlMonks website, the problem is a bad status return in the module when you use close() method, so the simple solution is to "undef $sth" at the end:
$sth->finish;
undef $sth;
$dbh->disconnect;

Thursday, October 8, 2009

Reading the DNA

Current DNA sequencing technologies are based on sequencing-by-synthesis or similar technics, where enzymatic reactions add one or more nucleic bases marked. But IBM is working in another way to know the composition of a DNA sequence, using nanotechnology and electronics to "read" base by base. No enzymes means faster and cheaper process. We're closer to the $1,000 genome ...



Source: http://www.engadget.com/2009/10/08/ibms-ultra-cheap-dna-transistor-dream-could-lead-to-personalize/

Friday, September 25, 2009

Wednesday, September 23, 2009

Tron Tributes

The next video is a recompilation of Tron-like scenes in movies, music videos, tv commercials, even Tron on Ice, turn on the speakers because the video music if a Daft Punk (who else?) remix.



I have the Tron screensaver in my Mac.

Monday, September 14, 2009

World twit "good morning"

Amazing video, please check the full details in their Vimeo page or in blog.blprnt.com.

GoodMorning! Full Render #2 from blprnt on Vimeo.

Thursday, August 27, 2009

Movie Time Travel

Information is Beautiful created this image, amazing and mind-blowing:


after admired it, please read the full description.

Monday, August 24, 2009

All begin with a Big Bang

The next video is a short explanation of how we think the Big Bang was.



I prefer the Tolkien's version, an universe created with a symphonic music, and of course, I love "The Big Bang Theory" tv-serie.

Tuesday, August 11, 2009

Jackie Chan and antivirus software

This video almost convince me to buy and install a MS-Windows machine, Tron anyone?

Wednesday, August 5, 2009

Supported Features

Image source: http://imgs.xkcd.com/comics/supported_features.png
Who wants to watch a movie when you're crunching numbers (at least until = 42)?

Monday, August 3, 2009

MS + Yahoo!

Source: http://www.denverpost.com/opinion/ci_12946782

Friday, June 12, 2009

Zero G

Last March, the Japanese astronaut Koichi Wakata joined the ISS, staying until few days ago. Besides his scientific task, he was doing a list of experiments for the people (selected in a contest), the list includes:

1. Calisthenics: Is it possible to follow an audio-guided workout program in zero gravity?
2. Backflips: On Earth, backflips take a lot of practice and leg strength. How about in zero gravity?
3. Volleying (soccer): Crumple a piece of paper into a ball and try kicking it around. How does the ball behave in zero gravity? Can you volley it?
4. Push-ups: In space, can you do push-ups while facing the ceiling or walls?
5. Cartwheels: In zero gravity, can you rotate yourself continuously like a windmill?
6. Swimming: Try to swim through the air as if you were in water. Can you move forward by swimming? If not, why not?
7. Spin like an ice skater: On Earth, ice skaters can increase their rotation speed by pulling their arms closer in to the body while they spin. Does the same thing happen in zero gravity? If so, what is the reason?
8. Folding clothes: In space, can you fold clothes and put them away as you do on Earth? It seems that the shirt sleeves would be difficult to keep in place. What is the best way to fold clothes in space?
9. Magic carpet: Try to sit on a floating carpet. Magic carpets are a fantasy on Earth, but are they possible in space?
10. Water gun: On Earth, if you squeeze a drink bag, a single stream of liquid shoots out through the straw hole and falls to the ground. How does the liquid behave in zero gravity?
11. Eye drops: On Earth, you have to face upward to put eye drops into your eyes. Is there a better way to do this in zero gravity?
12. Propulsion through space: When floating in zero gravity, how much power do you need in order to propel yourself around? Can you move simply by blowing air from your mouth or by flapping a hand-fan?

The next four activities are to be performed by two people:
13. Arm wrestling
14. Shaking hands
15. Sumo
16. Tug-of-war

He did all the list, and the results are really curious. The next is the video of the experiments, I cannot imagine how is to be in zero-G.

Wednesday, June 3, 2009

One year on Mac


I always admired Mac as computers, cool designs, functionality in the OS, *nix in the core, everything just works. Many years ago I wished to have my own Mac, unfortunately I never had the income to buy one (Mac are even more expensive in the third-world).

Twelve months ago I started to use a Mac for my everyday coding/writing work. Before this, I used Linux principally for the same tasks, so the migration was painless. My Mac is a iMac with a nice large screen (I love it!), 2GB RAM, 300 GB HD, Intel Core 2 Duo, very stylish. The exception in the design is the damn mouse, why one button? Many applications require the missing button, forcing to use the Control key, besides the scroll button is failing and failing. When I find a nice and cheap white 3-button mouse I change it.

Put the terminal in the bar as access was the first thing I did, I discovered that is a common pattern for coding people like me. I was not impressed with the desktop effects, after being a regular user of Compiz/Beryl/Matisse you can said: "I saw better useless effects in Linux".

My regular coding work is based in Perl, so I did not have any problems, but in other tools I can say the same. Mac updates release are good, but they aren't the last stable version in some cases, Java is a nightmare. I found some interesting programs to use/test based in Java, when I tried to run, many required the 1.6 release, Mac Leopard has the 1.5 version and update is not an option. Frustration!

Other bad point is the software update, after some time you install more and more programs, and keeping update is a waste of time, you can use some tool like AppFresh, but this are miles away from the versatility and utility of Linux application managers like dpkg, apt-get, urpmi, yum, ... which with a single and simple command can update (or upgrade) a full system.

I use a lot the shell for my work, from running programs to parse files, the mac has the typical bash shell and many of the *nix tools, but not everything, so I use Darwin Ports to install the missing commands (wget, imagemagik, ...).

My final claim is for iTunes, I commonly work hearing music (my nirvana state includes music/coffee/food), I have a large collection of music accumulated over the years (legally acquired, I own the CDs, stored in a big box in my home in Mexico). I obviously transfer my music to iTunes, and iTunes fails to recognize the MP3 tags, many of the MP3s are in wrong categories, the albums are split or files has a bad annotation. OGG files are not recognized and you need a special codec to use it and you need to convert the files to ACC before to transfer to iPod. Why does iTunes update need a system reboot? More frustrations!

After this time, I recognize the value of a Mac as workstation, specially because I used mine with really complex and heavy genomic/image analysis. And many tools/programs prepared for the Mac installs/works perfectly without problems. Now I'm using Mac specific programs and after 75% of disk usage I can't switch. I'm still asking me how this Mac looks/works with Mandriva.

I'm still believing the perfect system does not exist (at least for me).

Thursday, May 28, 2009

Analog random number generator


With my new projects I had to use -and abuse- of (pseudo)random generators. I remember my first classes of statistics when the teacher used simple examples of probabilistic events with coins or dices. After going serious in random generations, you can see a lot of research in this area. From computer algorithms to real probabilities, random is anywhere (specially in data validation, a common task in this new age high throughput biology).

The problem with random generators is you can never be sure is totally random, then some approaches use less sophisticated but effective random generators like this:



In this case, is a special machine for a games company named GamesByEmail.com, as you see, they automatically roll dices and take a picture, then a computer process the image and return the numbers.

I must confess, I have a 8-Magic-Ball (Eragon edition) in my desk to answer the important questions in my life.

Friday, May 22, 2009

Hi, real human interface

I like more the concept of a mouse inside the CPU, but this is a great video:

Hi from Multitouch Barcelona on Vimeo.

Wednesday, May 13, 2009

Installing Mandriva 2009.1 in the EeePC 1000H

Finally I have a netbook, the Asus EeePC 1000HA, a nice mini-laptop in white pearl.

The hardware specifications are:
  • Intel Atom 1.6GHz processor
  • 160GB 5400 RPM Hard Drive
  • 1GB of DDR2 RAM (667MHz)
  • 10” WSVGA LED-Backlit 1024 x 600 LCD
  • Ports: 3 USB 2.0, 1 VGA monitor out, headphone jack, microphone input, SD card reader (SDHC compatible), lock slot, Ethernet 10/100
  • Webcam (1.3 MP)
  • Speakers and microphone built-in
  • Battery: 7.4v 6600mAh 6-cell battery (3.5 - 7 hrs)
  • Wireless: 802.11b/g
  • Size: 25 cm (W) x 19 cm (D) x 3 cm (H)
  • Weight: less than 1.5 kg
The worst part is the SO, I don't understand why this model is exclusively sold with the almost-dead XP, so I installed the latest Mandriva release.

Because this system require an external drive to boot an installer, I download the image for the official mirrors (inside /MandrivaLinux/official/2009.0/i586/install/images/) named all.img, this is a prepared image to copy in a USB drive and includes a boot-loader and the basic tools for a network (NFS, FTP, HTTP, ...) or local installation from a drive. For more information go to: Mandriva Wiki.

To create the USB drive just use the dd command: dd if=all.img of=/dev/sdb1
(Caution: the data in the USB will be lost, and check the drive twice, don't delete your disk).

Then just put the USB and turn on the EeePC, the manual said you need to press ESC at boot to access the boot menu, wrong, this BIOS version requires you press F2 to access the BIOS configuration, I check the boot device order and put the external devices as first option. Wrong again, when you connect an USB storage drive, the BIOS assign it like a drive, so you need to switch the disk order (put the USB as first drive and the hard drive as second). I lost about a hour discovering this peculiarities.

After boot correctly the Linux installer, you need to select your installation method, I use HTTP from a Canadian mirror, the installer download the packages and start the installation. Of course the configuration is simple, I choose Gnome as desktop and manually partition my drive. The disk was already parted, the first partition is the Windows part (~70 GB), the second is an empty "user space" (~60 GB), the third is the Windows Restore part (~3 GB) and a small partition for the BIOS update.

I use the second partition with this table configuration:
  • / - Ext4 - 10 GB
  • SWAP 2 GB
  • /home - Ext4 - ~50 GB
The rest is to wait to the installation ends, few configurations (root password, users, time zone, boot loader) and you're done.

I rebooted the system and check if there are any problem: NONE, all the hardware is correctly recognized and configured, webcam, WiFi (with atheros drivers), kernel laptop mode, sound, graphics, screen, keyboard, touchpad, kernel sensors (temperature, battery, cpu-scaling) ...


The only problem I see is the sensibility of the touchpad, too much sensible, but it also happen in the XP, later I'll search the web for the xorg configuration and adjusted it.

I totally recommend Mandriva 2009.1 for this computer!

Monday, May 4, 2009

The biggest cults in tech

Infoworld reveals the biggest cults in tech:
  1. The Way of the Palm
    Established: 1996
    Gathering of the tribes: The Palm Forums
    Major deities: Jeff Hawkins, Donna Dubinsky
    Sacred relics: Pilot 1000 and Pilot 5000
    Mantra: The Pre will set us free
  2. Brotherhood of the Ruby
    Established: 1994
    Gathering of the tribes: RailsConf, RubyConf
    Major deities: Matz, DHH
    Mantra: MINSWAN (Matz is nice, so we are nice)
  3. The Ubuntu tribe
    Established: 2004
    Gathering of the tribes: Ubuntu Developer Summits
    Major deity: Linus Torvalds
    Minor deity: Mark Shuttleworth
    Animal spirit guides: Breezy badgers, dapper drakes, feisty fawns, gutsy gibbons, hardy herons, intrepid ibexes, jaunty jackalopes
  4. The Commodorians
    Established: 1982
    Gathering of the tribes: CommVEx, C4 Expo, World of Commodore
    Major deity: Jack Tramiel
    Minor deity: Jim Butterfield (1936-2007)
    Sacred relic: Commodore C65
  5. The Order of the Lisp
    Established: 1958
    Gathering of the tribes: International Lisp Conference
    Major deity: John McCarthy
    Minor deities: Paul Graham, Peter Norvig
    Holy Scripture: "Paradigms in Artificial Intelligence Programming"
  6. Monks of the Midrange
    Established: 1960
    Gathering of the tribes: Common 2009
    Major deity: Dr. Frank Soltis
    Holy scriptures: The IBM Redbook
    Sacred relic: Original AS/400
  7. The Tao of Newton
    Established: 1993
    Gathering of the tribes: Worldwide Newton Conference
    Major deity: John Sculley
    Minor deities: Too many to name; many are listed in MSU's unofficial Newton Hall of Fame
    Holy scripture: The Newton FAQ
    The Antichrist: Steve Jobs
  8. Windows
    Established: 1995
    Gathering of the Tribe: InfoWorld and other magazines that pretend that everything except Windows is a "cult"
    Major Deity: Bill Gates
    Sacred Relic: 30-letter authorization keys
    The Antichrist: Linus Torvalds
  9. Cult of free software
    Established: 1985
    Major Deity: RMS
    Sacred document: GPL
    Antichrist: !GPL'd software
  10. Cult of debian
    Established: 1993
    Major Deity(s): Bruce Perens & people called Ian
    Sacred relic: Debian 1.0 discs
    Antichrist: ubuntu
Note: 8, 9 and 10 are mentioned in SlashDot, I just added to the list.

I'm safe by now.

Wednesday, April 29, 2009

Swine flu

The world is in a critical time now, the swine flu is spreading fast, the biggest impact is my loved country Mexico, reporting thousand of cases and more than 150 deceases. Other countries have been reported positive cases around the globe.

Many things isn't clear, are the numbers real?, when did exactly this started?, why is this virus so different?, why are the cases in Mexico lethals?

A good review of the situation is in the Germán's Blog, he is a graduate student of the UNAM in the faculty of biomedics. (My apologizes, it's is in Spanish and I don't want to translate).

By the moment, you can watch the spread with this Google Maps Mash-up:

View H1N1 Swine Flu in a larger map

Remember, the best prevention is the hygiene and medial attention as soon as you feel bad.

Thursday, April 23, 2009

GeneTitan video

Yes, it's a not so good commercial, it shows the new Affymetrix GeneTitan with toys!

Tuesday, April 21, 2009

Scary Email



Just imagine when the spam arrives ...

Image from: http://i.gizmodo.com/5220568/holy-crap-email-was-scary-in-the-1970s

Sunday, April 12, 2009

Real science on TV


Image from: http://www.phdcomics.com/comics/archive/phd040609s.gif

I admit, some times I'm watching a TV shows and put a smile in my face when I catch some big mistakes in the scientific/technical plot. Anyway I'm a fan of science fiction series.

Saturday, April 11, 2009

Bigger tiny memory

Today I had received my new pendrive, I found it few days ago surfing in tech-news sites. It's amazing how technology decreases the size and increases the performance, for example I compare my old pendrive with the new:

As you see, the old drive is bigger as a finger, and the new one is half of this size, but with 32X more storage capacity and faster. The bad thing is I received it inside a big letter-bubble-envelope for this, I understand package protection but the size of the envelope is ridiculous. The good thing is the price, as much as a movie ticket.

Besides my Dropbox account, some times I need a bigger drive to move my data, I'm not the typical guy with Gigas of music, movies or pictures, when you work with large biological data as genomes, you never have enough space in your drive. Now my workstation is processing 70 Gb in images, I'm using a random sampling inside the images to reduce the computer time.

My kingdom for a personal supercomputer!

Wednesday, April 8, 2009

We're Linux video contest

The Linux Foundation launched the video contests for a "We're Linux", last day they announced the finalists, but I like this one (probably because my biological background):

I hope this wins.

Update: Origin is second place, the winner is http://video.linuxfoundation.org/video/1106

Tuesday, April 7, 2009

Green Supercomputing

There are 3 big problems when you want a supercomputer (besides the cost, of course), the first one is the space, a cheap and common approximation is to use a cluster design which you wire, as many as you can, independent computers. So you need to reserve an adequate space for the equipment, if you're not using servers racks, the space is a problem.

The second problem is the calorific power generated by the computers, a single desktop computer can be really hot in intensive use, so imagine when you put many machines in a same room together. I remember the first cluster of the bioinformatic lab, the provider installed the hardware one day before they installed the software, but when the guy turned-on the systems, all the RAMs were burned. The provider changed all the RAMs for a more heat resistant brand. After some time and after some expansions in the cluster, the cooling systems also was required to be changed for an industrial equipment, in summer it feels like an oven.

The third problem is the electricity cost, because many energy is loose in form of calorific transformation, the system is not efficient.

Last day reading the Icaza's blog, I noted a company named SiCortex, who is selling a green supercomputer solution, based in multi-CPU towers, they have from desktop size with 72 processors, a mid-size configurable with 324 to 1,458 processors and up to 1.9 terabytes of memory, and the "I-want-of-this"-size: the SC5832 with up to 8.2 TFlops of compute power in a single, self-cooling cabinet and configurations range from 1,944 to 5,832 processors and up to 8 terabytes of memory. I did not see prices, except for the desktop version which cost ~$24k usd (a special offer can reduce the price to $16k), I cannot imagine how much cost the others.


The secret is the tower design and the type of processors used, the system use very few space, few energy and don't need A/C. Of course it runs GNU/Linux (RedHat) and support common tools (GCC, Fortran, MPI, ...). For more technical details check the Architecture Tour.

Friday, April 3, 2009

Robot scientist

Amazing, I know about the use of robotics systems to manage high throughput experiments, for example when you need to prepare reactives dilutions in plate series of 96 or 384 (some time ago I hacked one of these equipments using reverse engineering and Perl), or picking bacterial colonies and extract the DNA.

Now a couple of reports in Science (this and this) are describing the use of a robotic system to study yeast mutants in media deprivation, and after run 1000 experiments every day, the system report some insight about the results. Check the video and more information in NewScientist.

Thursday, April 2, 2009

Face gestures in Opera

The best joke for April 1st in my opinion was this:

Friday, March 27, 2009

Robots

Since I've been working on high throughput biological image analysis looking for patterns and modeling 3D visualizations, I've been reading about computer vision. Beside the beauty of mathematical formulations for image processing, I found many related papers about robotics and the impressive development of robots which a specific use.

Reading this Gizmodo article I noticed the Hirose-Fukushima Robotics Lab at Tokyo Tech, check the movies section for examples of their fantastic robots like this:

video

I always like the robots in SciFi, from R2D2 to Megatron, but when you see the actual research in robotics, you can believe in a close future we'll have more sophisticated robots (complete robots, not only robotics arms or autonomous vehicles).

Tuesday, March 17, 2009

No miracles

Last weekend I saw this cartoon printed in a T-shirt at the PSC, and last day Kbyte's twitts reminded it to me.


Cartoon by Sidney Harris.

Saturday, March 14, 2009

Old computers Dream

I own a little old laptop (it was owned by a very good friend, traded in a perfect time for us). It's a HP pavilion ZT1114, PIII @ 1GHz, only 256 Mb in RAM, no battery, I recently change the HD. The computer is still working well, except for versions of heavy Linux can run but response time is slow.

Long time I used in this machine Mandriva, from 2007 to 2009 releases, changing the default desktop to XFCE to save memory and optimizing the system, also I tried Puppy Linux and other minimalistic Linux distributions. But always I feel something missing, an ultimate Linux distro with fast performance.

Until few weeks I noted the new release of DreamLinux, a Debian based distro with a pretty and fast desktop (XFCE). I had installed last week and after one week of use, I like it!
  1. Everything works well.
  2. The AWN bar looks like MacOS X.
  3. The Debian stability and large collection of packages.
  4. EasyInstall application is really useful.
  5. DCP-Control panel works well.
Of course not everything was perfect:
  • The first time I installed it, the main user account wasn't created with the correct configuration, so I logged as root, created a new user again and used it.
  • This new user, I added manually to sudoers list.
  • Wicd cannot use my WiFi configuration, it never connect to my router, so I wrote a shell script to activate my network.
This kind of problems scares a newbie, I could resolved, but other people just format the machine.

Some screens:



Saturday, March 7, 2009

Hitler disappointed with OpenGL

This a video so geek, but it's funny (if you known about OpenGL dev).




From: http://geekmx.co.cc/archives/57

Tuesday, February 24, 2009

DNA fractals

Some days ago, I had a talk with I who suggested me to use some mathematical approaches to compare artificial with real sequences (one of my projects). Besides mutual information and SVM classification systems, he told me about fractals presents in the sequences. The method had been called "Chaos game for DNA" (emboss has a subprogram to do it named chaos) and the algorithm is very simple:
  1. use a square with each corner represents a "letter" (AGCT),
  2. start in the middle of the square and read the sequence,
  3. you add a point in the halfway of the previous point and the new letter,
  4. continue until you finish the sequence

This method was previously described in the 90's, recently we have more genome information to continue exploring this characteristic in the DNA sequence, some application include alignment-free comparison and other iterative systems which reproduce this effect.

After I plot some sequences, I asked me to test other species to see differences, but as G concludes, the pattern is present because genomic sequence has low CpG content (this is "CG" are poorly represent, biological implications are DNA regulation with methylation).

We still discussing if DNA can be called "fractal" and if this characteristic can be used to scan sequences looking for functional/non-functional parts or some insight.

I twitted some thoughts about this like "fractals are beauty", Marcha replied: "fractals are evil", so "evil is beauty".

Thursday, February 19, 2009

Tips And Tricks For Bioinformatics Software Engineering

Gus told me about this blog post which is the condensed version of this presentation:
Let's discuss some points:
  1. Learn UNIX. This is basic, many people comes from the MS-world and wants to do the same tasks as in UNIX, wrong, the UNIX philosophy and design is optimized for heavy task and multiple activities, besides the complete support for programming, servers, administration, ... I say: don't waste your time, use UNIX (Linux, Mac OS, BSD, Solaris, ...).
  2. Know many computer languages but be master in ONE. In this part the talk is focused in Java/Python/Ruby, I still suggest Perl. Perl big problem is the object-oriented pragma, but nothing compares the power of regular expressions and text-parsing of Perl. Besides Perl is less grammar strict than Java/Python, it's easy to learn but easy to acquire bad habits.
  3. Don’t reinvent the wheel. I agree, why do spend time and money developing something which do the same as other existing tools but in different colors? Code less means faster development.
  4. Learn one text editor really well. You can write in many text editors, but few really support code development and other features which enhance your code-fu.
  5. Control version. CVS and similar tools helps to monitor, share and backup your valuable code.
  6. Don’t be afraid to use more than 3 letters to define a variable. Common mistake in beginners (and lazy programmers), you must have a legible code, always.
  7. Balance architecture and accomplishment. Software development can be elegant, complete, extensible and with order, but this takes time, find an equilibrium.
  8. Automate documentation. Or insert the documentation in the code, I use PerlDoc features in my code, easy way to write the help files.
  9. Kill the flat file. New technologies has high throughput files, so is better to use a DB (or kind-of) to store and retrieve the data. BerkelyDB, SQLite, SOAP/ReST, ...
  10. New ways to do parallel computing. Check MapReduce, cloud systems and support for multiple cores technologies.
  11. Embrace hardware. Read and understand novel technologies in data-crunching, vectorial CPU are back with graphical processors (GPU) and FGPA chips. Many algorithms can be ported and be faster in this chips, but not all of them.
  12. Playing nice with others. Support multiple outputs like XML, JSON, YAML to easy integrate with other tools.

Wednesday, February 18, 2009

Filtering common miRNAs

Last day, M asks me how to compact the microRNA mature fasta-file from miRbase, the problem is many miRNA are conservated across species, so the fasta file with all the mature sequences has a lot of redundancy and he wants unique sequences to screen some sequences.

This is a simple Perl solution, the first step is to read and collect the sequences, the trick is to save in a hash using the nucleic sequences as "key" and append the names as "value". After this, simply extract the "keys" and parse a little the "values" to have a common identifier.

Code:

#!/usr/bin/perl -w
use strict;

=head1 NAME

uniq_mir.pl FASTA

=head1 DESCRIPTION

Filter the multifasta file of miRbase (mature.fa)
into unique miR by sequence.

=cut

$ARGV[0] or die "Usage: uniq_mir.pl FASTA\n";
my $name = '';
my %seqs = ();
open F, "$ARGV[0]" or die "cannot open $ARGV[0]\n";
while (<F>) {
chomp;
if (/^>/) {
s/>//;
my @line = split (/\s+/, $_);
$name = shift @line;
}
else {
$seqs{$_} .= "$name:";
}
}
close F;

foreach my $seq (keys %seqs) {
my $sp = $seqs{$s};
my @sp = split (/:/, $n);
my $mir = shift @sp;
$mir =~ s/^\w+-//;
$sp =~ s/:$//;
print ">$mir $sp\n$seq\n";
}

=head1 AUTHOR

Juan Caballero @ 2009

=head1 CONTACT

linxe11 _at_ gmail.com

=head1 LICENSE

Perl Artistic http://perldoc.perl.org/perlartistic.html

=cut

Thursday, February 12, 2009

Command-line Fu


Recently I found this great site Command-line Fu, a site where you can find these beautiful gems for the command line but it's different from other collections:

What's this?

Command-Line-Fu is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on and discussed - digg-esque voting is also encouraged so the best float to the top.


So, anyone can share their commands, you can look, learn and use or suggests better ways to do it, and beyond the Bash, also include tags (functions) for one-liners scripts in perl, python, etc. or by specific commands like curl, ssh, grep, ...

With Twitter and RSS report systems you can follow the latest commands too (but only report the link and the function, not the command).

Sweet.

Via: DownloadSquad

Monday, February 9, 2009

1234567890


The next February 13th has a special event, not for everyone, just for people like us (yes, you know, accept it ...) because the UNIX time at 11:31:30pm UTC will read in the terminal in curious form: 1234567890.

Why? Check Wikipedia:
"Unix time, or POSIX time, is a system for describing points in time, defined as the number of seconds elapsed since midnight Coordinated Universal Time (UTC) of January 1st. 1970, not counting lap seconds. It is widely used not only on Unix-like operation systems but also in many other computing systems. It is neither a linear representation of time nor a true representation of UTC (though it is frecuently mistaken of both) as the times it represents are UTC but it has no way of repesenting UTC leap seconds (e.g. 1998-12-31 23:59:60)."

If you want to check it, or see other values for your favorite number, you can use the Perl one-line:

perl -e 'print scalar localtime(1234567890),"\n";'

Note: image from Wikimedia showing when the 1000000000 seconds since epoch was celebrated at a party held by DKUUG in Copenhagen, Denmark 2001-09-09T03:46:4.

Friday, February 6, 2009

People loves Win7 ... wait it's KDE4!

ZDNet Australia test some people in the street, they showed a Linux (which one? they didn't mention ...) with KDE4, they activate some fancy effects and told the people "this is the new windows 7", ... many people like it!



I think they must tell the true, to see if they want to switch to Linux, anyway the cheaper W7 will be $200 usd (I prefer to buy a new computer with this money, I'm still saving for my AAO, donations accepted).

Wednesday, February 4, 2009

Bigger, Faster, ...


IBM announced the new big project to build the next #1 supercomputer of the world, codename Sequoia, it's impressive the limit of 20 petaflops ... that's more than the combined power of all top 500 supercomputers (#1 is only 1 petaflops).

The Sequoia will be powered by 1.6 million cores (specific 45-nanometer chips in development) and 1.6 petabytes of memory. It will be housed in 96 refrigerators spanning roughly 280 square meters.

Of course the owner will be the USA gov, who's else?

From: Gizmodo
Image from: xkcd

Monday, February 2, 2009

Bash scripting tips

Bash is almost a complete programming language, I use every day (switching from the default tcsh in the servers, in Mac OS X is default) for automatic job launching.

Extracted from: http://hacktux.com/bash/script/efficient.

  1. Avoid full paths to bash built-ins. Except when you require specific commands, when I use systems with 32 or 64b binaries, I need to specify which one to use, or define an specific directory to use in the PATH or as a variable.
  2. Avoid external commands for integer math. Because bash has math evaluation.

  3. Avoid using cat. Commonly people use cat and send to other commands with a pipe, but many commands can read files in the parameters.

  4. Avoid piping grep to awk. Because awk can filter the inputs, use /pattern/

  5. Avoid piping sed to sed. Multiple filters can be applied to the input with the -e option ( sed -e "s/this/that/" -e "s/old/new/" filename ).

  6. Use double brackets for compound and RegEx tests. [[ ]] evaluate the operations between, ( if [[ expr1 && expr2 ]]; then something; fi )

  7. Use functions for repetitive tasks. Define some functions is simple ( function () { do_something; return $? } )

  8. Use Arrays Instead of multiple variables. Yes, bash has array support, declare as normal variables, and access with []. ARRAY = ("one", "two"); echo ${ARRAY[0]}

  9. Use /bin/mktemp to create temp files. This is new for me, good to know. tempfile = /bin/mktemp

  10. Use /bin/egrep or /bin/sed for RegEx pattern matching. Only for basic RegEx, for more complex patterns it's better to use Perl.


Friday, January 16, 2009

Metric


I need the inverse table ... why are they still using it?
From: http://imgs.xkcd.com/comics/converting_to_metric.png