Quickly navigate your filesystem from the command-line

I have been interested to the topic Data Science for a while. Today, I was looking for any book on this topic and I came across this one Data Science at the Command Line. The book is not published yet and I can read it through Safari Books Online. I then came across the writer's blog. Among many things, I found this page http://jeroenjanssens.com/2013/08/16/quickly-navigate-your-filesystem-from-the-command-line.html very very interesting. Since I found it so useful, I felt mentioning it here. All credit goes to Jereon Janssens. 

avoid strange characters for FTP password

I recently created a ftp server on Amazon created strong password (LAl)#oT!qjyQT!:@*)for a user's login.  Though I could login using that credential using FileZilla, the user was not having success. He was seeing the following error:

Status:                  Resolving address of ftp.***.com

Status:                  Connecting to *.*.*.*:21...

Status:                  Connection established, waiting for welcome message...

Response:            220 (vsFTPd 2.2.2)

Command:          AUTH TLS

Response:            530 Please login with USER and PASS.

Command:          AUTH SSL

Response:            530 Please login with USER and PASS.

Error:       Critical error: Could not connect to server

It had turned to be encoding issue. I found this good explanation here which made things clear for me. 

The lesson for is, keep ftp passwords simpler in future.

Solving MySQL error 1062

While MySQL replication is running, it might throw errors like below:




140502  1:56:38 [ERROR] Slave SQL: Could not execute Write_rows event on table DB_name.table_name; Duplicate entry '138114741' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000054, end_log_pos 880389787, Error_code: 1062

Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'mysql-bin.000054' position 880388660

Sometimes it happens that the MySQL replication stops because some statements received from the master fail when replayed on a slave. Luckily, however, this isn’t always the case as sometimes the failing statements don’t affect the data or might only affect data that isn’t important or is removed frequently (for example tables containing temporary data); in such cases then you may not need to rebuild the slave or re-sync with the Percona tools (which perform a time consuming comparison operation -especially on large tables- and might cause high load on both master and slave, up to causing locks and various issues depending on the usage of the servers at the time and the resources available); you can instead get the slave up and running again very quickly by simply skipping the failing statements, provided that these statements can be safely ignored as well as any data they might affect.

This can be done by temporarily stopping the slave, then instructing MySQL to skip one or more statements received from the master, and eventually starting the slave again.

mysql> stop slave;

Query OK, 0 rows affected (0.10 sec)

 

mysql> set global SQL_SLAVE_SKIP_COUNTER = 1;

Query OK, 0 rows affected (0.00 sec)

 

mysql> start slave;

Query OK, 0 rows affected (0.00 sec)




But the ultimate fix would be to replicate the faulty table from other DB server

Mysqldump:

mysqldump -u user -p DB_name table_name > table_name.sql

Restore:

mysql -u user -p db_name < /path/to/table_name.sql

Use of ProxyPassMatch

So, what to do if you need make use of Regular Expression instead of simple prefix matching.

ProxyPassMatch ^(/.*\.phtml)$ http://lt-legacy-1171567772.eu-west-1.elb.amazonaws.com$1

As you see here, any match for .phtml file will be passed to another backend server.

Forwarding Requests to Another Server in Apache

I have a server on AWS which runs php5. Client requests are sent to this server as it sits behind the ELB.  I also have another server which runs on php4 and isn't accessible publicly.  So, in this case, Client requests for contents written in php4 can only be accessible via the php5 server. To solve this issue, I used ProxyPass and ProxyPassReverse in apache configuration.


ProxyPreserveHost On
ProxyPass /maps/ http://lt-legacy-1171567772.eu-west-1.elb.amazonaws.com/maps/
ProxyPassReverse /maps/  http://lt-legacy-1171567772.eu-west-1.elb.amazonaws.com/maps/

So, requests to URLs starting with /maps/ to be forward to the server http://lt-legacy-1171567772.eu-west-1.elb.amazonaws.com/maps/.

The ProxyPassReverse directive ensures that any redirect headers sent from the php4 server (in this case lt-legacy-1171567772.eu-west-1.elb.amazonaws.com) wil be modified so that they appear to come from the php5 main server.

As you can see, the php4 server isn't actually the aws server itself. It's the AWS ELB which forwards requests to the backend php4 server.

If your backend servers sit behind a firewall/Security Group, you would have to open ports (http in this case) for the frontend servers (php5 server in this case).

Use hotelmap for London

http://www.hotelmap.com/ is a unique way to find the available hotels of your choice. I came across this through http://www.londontown.com/ and wanted to share with you all.

Hotelmap uses a referral system http://www.hotelmap.com/Refer. So you can also earn money by referring it to your friends and colleagues who are coming to London.

Port forwarding issue

Someone on ServerFault asked the following question for which I gave the answer. It was later accepted by the questioner. 

=====================================================================

So, the question was:

I have a Router and four computer connected as the local network.

IPs:

Router(192.168.8.1)(internet ip:125.119.203.127)
      server1(192.168.8.3)
      my computer(192.168.8.8)
      ......

The server1 runs on the RedHat Linux which is set to use the port-redirect, to redirect port 80 to 8081. While the tomcat in this server use the port 8081.

[root@server1 bin]# iptables -t nat -vnL
Chain PREROUTING (policy ACCEPT 3169 packets, 274K bytes)
 pkts bytes target     prot opt in     out     source               destination
    4   208 REDIRECT   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:222 redir ports 22
  221 14971 REDIRECT   tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:80 redir ports 8081

Chain POSTROUTING (policy ACCEPT 2253 packets, 141K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 2252 packets, 140K bytes)
 pkts bytes target     prot opt in     out     source               destination

Now in my computer, I can access the site in the local network use browser:

http://192.168.8.3
http://192.168.8.3:8081

The Router is configurated to redirect the port 80 to port 80 on 192.168.8.3

But I can not access the site using the internet address:

http://125.119.203.127

Normally, this request should be dispatched to server1 192.168.8.3:80, then server1 will redirect to port 8081.

Then I change the configuration of the Router, to redirect the port 80 to port 80 on my computer.

Then I can access: http://125.119.203.127

So what is the problem?

=====================================================================

For which, my answer was:

From outside the network use namp to check whether port 8081 is available externally

$ nmap -PN external_ip

This command will show all listening services externally. Since you can access the site using internally http://<internal_ip>:8081, the issue is with the router.

As you have mentioned that "The Router is configurated to redirect the port 80 to port 80 on ", what configuration is done to forward port 8081 from the router to the internal server?

I think you need to forward port 8081 from the router to port 80 to the internal server.