Mybookshelf2 (MBS2) is basically in beta stage and provides all functionality available in previous version and more. As I went through programing the new application using several new technologies, I’ve realized few things, which I’d like to share here. Continue reading Lessons Learned From Current Web Application Development
While SqlAlchemy (and Flask-SqlAlchemy) provides an easy way to create DB schema from scratch, migration of an existing schema is bit more challenging. As soon as you change attributes in your declarative ORM models, the underlying DB schema is to be changed for application to work. If you need to keep data , you’ll need to modify DB schema with some DDL commands (ALTER TABLE …) and this functionality is not part of SqlAlchemy nor Flask-SqlAlchemy. Continue reading Easy SQL Schema Migration for SqlAlchemy and Flask
Mybookshelf2 enters the stage, when code can be considered of alpha quality. Basic functionality is there, so it could be tried and I plan to move my ebooks collection to it soon. MBS2 is packed with new technologies and comparing to previous version (Mybookshelf) it can be considered a completely new application.
So what’s new? Concerning functionality not much, but a lot happened internally. For me it’s more about testing and playing with new technologies then managing ebooks (which is still important of course). Maybe some of currently used technologies are bit of overkill (like using WAMP protocol), but nevertheless they play important part in the application. Continue reading Mybookshelf2 Alpha Version is available
While Gnome and it’s derivatives support automatic proxy detection, it do not work well for all programs, particularly for command line programs. I’ve found that using simple script in /etc/NetworkManager/dispatcher.d works better for me, which sets and unsets fixed proxy works better. NM dispatcher scripts are run each time network connections change (network up, down, VPN connect etc.) and received two parameters ( interface name and status) and bunch of environment variables. Continue reading NetworkManager Script to Set HTTP Proxy
File operations and other IO operations can block asyncio loop and unfortunately python does not support true asynchronous disk operations (mainly due to problematic state of async disk IO in underlying os – aka linux – special library is need for true asynchronous disk operations so normally select (or other IO event library) always reports file as ready to read and write and thus file IO operations block). Current solution is to run such operations in thread pool executor. There is asyncio wrapper library for file object – aiofiles, but there are also many blocking functions in other python modules – like os, shutil etc. We can easily write wrappers for such methods, but it can be annoying and time consuming if we use many of such methods. What about to write a generic proxy, which will assure that methods are executed in thread pool and use this proxy for all potentially blocking methods within the module. Continue reading Asyncio Proxy for Blocking Functions
My actual master studies topic was AI (more then 20 years ago). Artificial Neural Networks (ANNs) were already known and popular branch of AI and we had some introduction to basics of artificial neural networks (like perceptron, back propagation, etc.). Though it was quite interesting topic, I had not seen many practical applications in those days. Recently I’ve chatted with old friend of mine, who stayed in university and is involved in computer vision research, about recent advancements in AI and computer vision and he told me that most notable change in last years was that neural networks are being now used in large scale. Mainly due to increase in computing power neural networks now can are applied to many real world problems. Another hint about popularity of neural networks came from my former boss, who shared with me this interesting article – about privacy issues related to machine learning. I’ve been looking around for a while and it looks like neural networks are becoming quite popular recently – especially architectures with many layers used in so called deep learning. In this article I’d like to share my initial experiences with TensorFlow, open source library (created by Google), which can be used to build modern, multi-layered neural networks. Continue reading Revival of Neural Networks
- get suggestions from server via REST API (JSON payload)
- simple, yet flexible (value can be calculated from received suggestions)
- flexible ways to display suggestions (ideally provide a template to display suggestions values)
- suggest values as one types matching starting letters (case insensitive and diacritics insensitive)
- cache server responses for efficiency
- can use both mouse and keys (arrows + enter) to select suggestion
I’m sure that existing autocomplete components like typeahead or JQuery UI Autocomplete would serve my purpose quite well. It’s fairly easy to integrate existing components from other frameworks into Aurelia (see this post for instance). But I decided to create my own, using only Aurelia (and a bit of JQuery – but it can be easily rewritten in pure DOM API – JQuery is used just for convenience, because it’s anyhow used in Aurelia skeleton app). I though it would be nice learning exercise (and it really was) and also we programmers (especially leisure programmers like myself) do like to reinvent the wheel, right? (But it’s not always bad idea – image world when only one type of wheel exists – by recreating existing solutions, improving them, changing them we can also achieve progress – Web UI frameworks themselves are good example of such progress). I’d like to share my experiences further in this article. Continue reading Next Adventure in Aurelia – Autocomplete Component
If you look for WAMP abbreviation over Internet, you will probably find that WAMP = Windows + Apache + MySQL + PHP – which was popular web stack some time ago (but who wants to run web server on Windows today? And other components now also have viable alternatives). But in this article I’d like to talk about WAMP = Web Application Messaging Protocol. WAMP is available as WebSocket subprotocol, but also can work on plain TCP or Unix domain sockets. Continue reading WAMP Is WebSocket on Steroids
Virtualenv is a must have for python development. If your project is a complex beast consisting of multiple services/components you want them see running in different terminals (ideally tabs of one terminal window). Staring all terminal manually could be cumbersome. This simple script starts terminal tabs (in gnome-terminal) with activated virtual environments and eventually appropriate services/applications started:
--tab -e "bash --rcfile $VENV -ci 'cd asexor && crossbar start'" \
--tab -e "bash --rcfile $VENV" \
--tab -e "bash --rcfile $VENV" \
--tab -e "bash --rcfile $VENV"
-ci argument – interactive shell must be enforced to run command with virtual environment loaded.w
Also gnome terminal recently drop support for –title parameter, which enabled to set title to the tab (really do not understand why, because it was very useful). So now our tabs will have same prompt.
This can be somehow fixed with modification of virtualenv activate script to include terminal escape sequence shown below (thus we will see current terminal directory as tab title):
Python 3.4+ provides excellent Asyncio library for asynchronous tasks scheduling and asynchronous I/O operations. It’s similar to gevent, but here tasks are implemented by generator based coroutines. Asynchronous I/O is useful for higher I/O loads, where it usually achieves better performance and scalability then other approaches (threads, processes). About a year ago I played with OCaml, where light weight threads/ coroutines and asynchronous I/O approaches are also very popular (Ocaml has same limitation for threading as Python – a global lock) and there were two great libraries – lwt and core async. Both libraries use monads as a programming style to work with asynchronous tasks. In this article we will try to implement something similar on basis of asyncio library. While our solution will probably not provide “pure” monads it’ll still be fun and we’ll learn something about asyncio. Continue reading Functional Fun with Asyncio and Monads