Friday, September 7, 2012

Filtering AppEngine dev_appserver logs on unix bash


Just a small post to make a note of something effective that I use. While working with the appengine, one often has to wade through lots of log statements, and having a lot of log statements is a good thing in my opinion. However, one can miss crucial information when the output isn’t very readable. I use the following command to filter and also color the output on bash shell on my Ubuntu machine.

dev_appserver.py . 2>&1 | grep --line-buffered -v -e 'GET /static' -e 'POST /stats' -e 'GET /favicon.ico' 2>&1 | egrep --color=always '^|WARNING.*|ERROR.*'

A little explanation for those who might find it helpful …
dev_appserver.py .: I prefer to run the app from within its directory.
2>&1: appengine puts out the log on stderr, and I’m redirecting it to stdout.
--line-buffered: because otherwise it won’t flush the output to the screen for every line.
grep -v: to show lines except that matching the lines that I don’t want.
WARNING.*: so that it colors the line up to the very end.

Creating an alias

Now, I surely wasn’t going to type all of that out each time. So I created an alias for it. For my bash shell, I added a line in ~/.bashrc as below:

alias da=”dev_appserver.py . 2>&1 | grep --line-buffered -v -e 'GET /static' -e 'POST /stats' -e 'GET /favicon.ico' 2>&1 | egrep --color=always '^|WARNING.*|ERROR.*'”

That worked well, until I wanted to add parameters to the alias, so that I could occasionally do a --clear_data to start with a fresh datastore. However bash does not support parameters to alias. So currently, this is what I have in my .bashrc.

myda() {
dev_appserver.py $1 $2 2>&1 | grep --line-buffered -v -e 'GET /static' -e 'POST /stats' -e 'GET /favicon.ico' 2>&1 | egrep --color=always '^|WARNING.*|ERROR.*'
}
alias da=myda

So now I can do this on the command line:
da .: to start the dev_appserver in the current directory.
da --clear_data .: to clear the datastore and start the dev_appserver in the current directory.

Hope that is useful for you!


9 comments:

  1. I stood motionless for long years with windows, as3 and PHP. But I still want to progress. Like this song: too old to rock and roll but too young to die. Is possible colorize log mesages? Again: congratulations for the initiative. it's a great incentive for beginners

    ReplyDelete
  2. Great and simple post you shared. Thanks for pointing that in your post.

    http://www.digitalbrief.com

    ReplyDelete
  3. Thanks for your article! I have been looking for quite a long time and fortunately I read this article! I wish you would continue to have valuable articles like this or more to share with everyone!
    abcya

    ReplyDelete
  4. Thanks for sharing this valuable information to our vision. You have posted a trust worthy blog keep sharing.
    procrastinate definition

    ReplyDelete
  5. I was very impressed by this post, this site has always been pleasant news. Thank you very much for such an interesting post. Keep working, great job! In my free time, I like play game: vex3game.com. What about you?

    ReplyDelete
  6. This article is really fantastic and thanks for sharing the valuable post.
    run3unblockedgame.com

    ReplyDelete
  7. I like your all post. You have done really good work. Thank you for the information you provide, it helped me a lot. I hope to have many more entries or so from you.
    Very interesting blog.
    www.baixarfacebookgratis.com.br

    ReplyDelete

If you think others also will find these tutorials useful, kindly "+1" it above and mention the link in your own blogs, responses, and entries on the net so that others also may reach here. Thank you.