sartak

sartak http://sartak.org Why I Read Books on My iPod http://sartak.org/2011/09/why-i-read-books-on-my-ipod.html <h4>I can copy Japanese sentences I understand into my Anki flash card deck:</h4> <img src="http://sartak.org/misc/blog/ipod-reading/copy-sentence.png" alt="" /> <img src="http://sartak.org/misc/blog/ipod-reading/paste-sentence.png" alt="" /> <hr /> <h4>I can copy Japanese words I don't understand into a dictionary: <a href="#fn-1">1</a> </h4> <img src="http://sartak.org/misc/blog/ipod-reading/copy-word.png" alt="" /> <img src="http://sartak.org/misc/blog/ipod-reading/paste-word.png" alt="" /> <hr /> There are a few reasons why I'm pretty excited about the iPhone 5 announcement. There will supposedly be a fat home button which allows gestures for easy switching of apps. If that feature indeed lands on the new iPhone, it will save me a couple moments and clicks, thousands of times over, as I flip between iBooks, Anki, and 大辞林 while studying. Also I'm still rocking a third-gen iPod Touch which does not have anything near a retina display, so that'll be great for turning those kanji from muddy to crisp. I don't foresee myself getting a new Kindle any time soon, even though those Kindle Touches look pretty damn good. <div class="footnotes"> <hr /> <ol> <li id="fn-1"> iBooks has a built-in dictionary, but it's not useful for my study because it's English-only. One way to make that button universally useful would be to let you set it up to launch some other application, but unfortunately built-in apps typically don't give you that much rope. Update: a reader has pointed me to iBunkos which is designed for reading Japanese text. iBunkos not only gives you the rope to launch the URLs of your choosing (which means you can launch any app that supports being launched to), but also has a builtin dictionary and a builtin link to 大辞林. Perfect! <a class="footnoteBackLink" href="#fnr-1">↩</a> </li> </ol> </div> 2011-09-28 TODO? vim ♥ ack http://sartak.org/2011/09/todo-vim-ack.html One pattern I frequently find myself following is acking for a very specific set of results... <pre class="code_snippet">ack -a 'uri_for.*(contact|address|phone)'</pre> Then I launch vim to start editing the files that were matched here... <pre class="code_snippet">vim `ack -la 'uri_for.*(contact|address|phone)'`</pre> Then I rewrite the search term using vim's regular expressions instead of Perl's far superior syntax... <pre class="code_snippet">/uri_for.*$contact\|address\|phone$</pre> Then finally I get along with my editing, using both <code>n</code> and <code>:bn</code> to advance to the each match as needed. It's generally at the "mentally translate the Perl regex to vim" stage where I just say fuck it and fudge it a little bit. There's gotta be something better here. How do you do this? Do you somehow have an editor that works better for this than mine? Surely there's room enough in this world for an ack + vim lovechild that not only loads the proper files into vim, but also automatically starts you off with the (translated!) regex in your search buffer, cursor already waiting at the first result. Do I have to write this thing myself? Update: apparently the answer is no! <a href="http://twitter.com/trs">Thomas Sibley</a> wrote a new tool called <a href="https://github.com/tsibley/viack"><code>viack</code></a> and I'm happy with how simple it is. \v gives a close enough approximation of Perl regexes. 2011-09-20 July 2011 http://sartak.org/2011/08/july-2011.html A telling side effect of my month of doing (mostly) nothing but studying Japanese: <img src="http://sartak.org/misc/blog/july2011.png" /> 2011-08-01 ニートであることについての記述 http://sartak.org/2011/07/ãƒ‹ãƒ¼ãƒˆã§ã‚ã‚‹ã“ã¨ã«ã¤ã„ã¦ã®è¨˜è¿°.html お久しぶりですね！！今日の多読を休止しようと思ったが、日常の出来事についてツイートより長い説明を書く。 第一に、先月に会社を辞めた。Infinity Interactiveに雇われた！<a href="http://moose.perl.org">Moose</a>とか<a href="http://www.iinteractive.com/kiokudb/">KiokuDB</a>とか<a href="http://beta.metacpan.org/module/Path::Router">Path::Router</a>とか<a href="http://beta.metacpan.org/module/Bread::Board">Bread::Board</a>をリリースした会社なんだ。ワクワクしながら始まるのを待ってる！まだ始まらなかった理由は、まる1ヶ月の休みを取りたかったんだ。7月中に激しく日本語をレベルアップしたいんだぜ！ 先週に<a href="http://www.yapc2011.us/yn2011/">YAPC::NA</a>を参加して、2つの演説(?)やった。1つ目は、<a href="http://perldoc.jp/docs/modules/Pod-Cpandoc-0.09/Cpandoc.pod">cpandoc</a>についてLTだった。インターネットが必要だった生の実演で、cpandocを使う方法を教えたが、楽しかった！2つ目は、<a href="http://beta.metacpan.org/module/Announcements">Announcements</a>入門を20分話した。YAPCで、miyagawaさんが僕に「How does it feel to be a ニート?」って冗談に聞いたので、このブログの表題をくれた。 最近、DTraceを習ってみた。強力すぎなツールだから、使わなくてはいけない。printf式でのデバグが好きなプログラマー、DTraceを調査して下さい！<a href="http://pinboard.in">Pinboard</a>でDTraceを記述する記事を集めている: [<a href="http://pinboard.in/u:sartak/t:dtrace">すべて</a>] [<a href="http://pinboard.in/u:sartak/t:%E6%97%A5%E6%9C%AC%E8%AA%9E/t:dtrace/">日本語だけ</a>]。PerlはDTraceのプローブがあるけど、関数の呼びと関数の返ししかトレースできない。しかし、僕がPerlにプローブ追加しようとしてる。一回目に、<code>${^GLOBAL_PHASE}</code>というの変数(<a href="http://perldoc.perl.org/perlvar.html#%24%7b%5eGLOBAL_PHASE%7d">perldocを参照</a>)の変化を知らせたい。それだから、p5pに<a href="https://rt.perl.org/rt3/Public/Bug/Display.html?id=94234">パッチ</a>をあげた。次のプローブを追加するのは、文字列式のeval、たぶん。今年のYAPC::Asiaで、僕の話はDTraceのテーマと思う。<a href="http://pghpw.org/ppw2011/">PPW</a>も、もし僕が行ったら。 次回に、最新の国語の勉強を物語る。さらばだ！ 2011-07-08 End-of-Line Whitespace in Vim http://sartak.org/2011/03/end-of-line-whitespace-in-vim.html Whitespace characters at the ends of lines are sloppy and (almost completely) useless. Not to mention when you notice and remove them, they clutter up your version control history. That can be mitigated by using <code>git diff --ignore-space-at-eol</code>, but it's better to never let it become a problem. So I have two settings in <a href="https://github.com/sartak/conf/blob/master/vimrc">my vimrc</a> that help me avoid committing EOL whitespace to any of my hobby or work projects. The first one highlights EOL whitespace so you can tell that it's even there, but in a way that isn't obnoxious. Vim does offer a builtin option to do this. But it sucks. <pre class="vim code_snippet">set list set listchars=trail:. </pre> These two settings make it so when whitespace occurs at the end of a line, each character is displayed as a blue period. This is good in theory, but actually it means that whenever you're typing new content, these blue periods show up every single time you're done typing a word and continue on to the next one. This is what I mean by obnoxious. Of course there's going to be lots of fleeting EOL whitespace if I'm typing a sentence or a line of code! So a few years ago <a href="http://tozt.net">Jesse Luehrs</a> and I banged our heads against our own separate walls until we came up with something that worked for us. I haven't seen this used anywhere else but it works perfectly for us so I wanted to share it. <pre class="vim code_snippet">autocmd InsertEnter * syn clear EOLWS | syn match EOLWS excludenl /\s\+\%#\@!$/ autocmd InsertLeave * syn clear EOLWS | syn match EOLWS excludenl /\s\+$/ highlight EOLWS ctermbg=red guibg=red </pre> What this does is highlight EOL whitespace with a red background except on the line you're editing. Which means it's not obnoxiously telling you about every time you hit the space bar. The highlighting only occurs when you leave insert mode or if you move the cursor to a different line, such as by hitting enter. Which we've found to be exactly the kind of highlighting we want. The <code>syn clear EOLS</code> is needed so when you switch modes, the appropriate syntax highlighting rule runs instead of both of them. The first line has two components of abominable Vim-specific regex: <code>\%#</code> matches cursor position, and <code>\@!</code> is like a negative lookahead from Perl for the previous atom. So in effect this matches end of line whitespace, except when the cursor is inside that whitespace. The other tool I use to combat EOL whitespace is to unceremoniously execute it. <pre class="vim code_snippet">nmap <leader>w :%s/\s\+$//<CR>:let @/=''<CR> </pre> This sets up a new normal-mode command <code><leader>w</code>. The <code><leader></code> is like a user-specific namespace for custom commands; for most people it's going to be <code>,</code> but I have it set to <code>\</code>. The <code>:let @/=''</code> bit empties the <code>/</code> register so that the <code>/\s\+$/</code> regular expression is not used for the <code>n</code> command or <code>hlsearch</code> highlighting. Unfortunately you lose whatever was in <code>@/</code> before you ran <code>\w</code> but that kind of problem hasn't affected me in practice. Some people run a whitespace stripper like this in a <code>BufWritePre</code> autocommand. But I don't like that solution because sometimes whitespace at the end of a line is important — such as in <a href="http://daringfireball.net/projects/markdown/syntax#p">Markdown</a>. Instead, the configuration I've described gives you tools for dealing with EOL whitespace sanely. Got any more? Several people on <a href="http://www.reddit.com/r/vim/comments/fvj70/detecting_and_deleting_endofline_whitespace/">reddit</a> have offered alternate highlighting solutions. Update - I'm stealing reddit user Amablue's <a href="http://www.reddit.com/r/vim/comments/fvj70/detecting_and_deleting_endofline_whitespace/c1j0uiy">improvements</a> to the whitespace stripper because it no longer stomps on <code>@/</code> and it reinstates the cursor position. I also like his <code><leader><space></code> over <code><leader>w</code>. <pre class="vim code_snippet">function! <SID>StripTrailingWhitespace() " Preparation: save last search, and cursor position. let _s=@/ let l = line(".") let c = col(".") " Do the business: %s/\s\+$//e " Clean up: restore previous search history, and cursor position let @/=_s call cursor(l, c) endfunction nmap <silent> <Leader><space> :call <SID>StripTrailingWhitespace()<CR> </pre> 2011-03-01 Interhack and TAEB Postmortem http://sartak.org/2011/01/interhack-and-taeb-postmortem.html <h5><a href="http://interhack.us">Interhack</a></h5> Interhack, a "smart" telnet client for server <a href="http://www.nethack.org/">NetHack</a> play, initially started because the annual <a href="http://nethack.devnull.net/">/dev/null</a> tournament didn't have any of <a href="http://alt.org/nethack">nethack.alt.org</a>'s interface patches - most importantly <a href="http://bilious.homelinux.org/?45">Color HP Monitor</a>. In vanilla NetHack, your HP display doesn't change color as you take damage, so in the tournament we would simply not notice we were being slaughtered because we had been subconsciously trained to notice damage using only our peripheral vision, which detects changes in color far more effectively than changes in numbers. To avoid rewriting history, I should note that in <a href="http://interhack.us/genesis-irc.txt">the IRC logs from Interhack's inception</a> (I'm in there as Eidolos), someone else named Stevie-O first wrote his own similar tool that made it obvious to him that his character was growing hungry. But we quickly took the idea (but importantly, neither the code nor the name) and ran with it. Interhack developed many more features beyond what we initially brainstormed but it was never designed to be driven by some other code. Interhack was always merely user interface enhancements. What was cool about the project was that such enhancements did not require patching NetHack, so Interhack could be used on NAO and /dev/null. Also, I only added (or permitted) features that would not take actions on the user's behalf. I was worried that such actions could misfire because of bugs in Interhack and cause someone to lose their game. In which case they'd have every right to be angry at me personally. In all of Interhack's usage I only heard of a couple of incidents where Interhack screwed someone up badly. In one, the person didn't get their wish because of a bug in Interhack's accidental-wish-escape protection. When you press an arrow key or other non-character keys, NetHack sees that as a sequence of characters beginning with escape, so it thinks you're trying to cancel your wish, which earns you a random item instead of a really powerful, useful one. We added code to catch that escape key and prompt you, asking if you really did mean to cancel your wish, which was certainly useful and saved wishes that would have been lost, but for whatever reason in that one instance Interhack hung at that prompt, so he had to kill it. When he restarted NetHack, which doesn't keep enough state in its save file to remember that you were at a wish prompt, it came back up with a wasted wish. Oops! <img src="http://interhack.us/img/wish-escape.png" /> Interhack lost steam because of a poor implementation (<a href="http://perldoc.perl.org/functions/do.html"><tt>do FILE</tt></a> is entirely insufficient for a plugin system. No, really.) and NetHack bot development. <h5><a href="http://taeb.sartak.org/">TAEB</a></h5> TAEB started as my <a href="http://taeb.sartak.org/history.html">fifth or so NetHack bot</a>, this time with a thoughtful design instead of just winging it (see above). Because as designed TAEB's AI was separate from the rest of the codebase, and thus very trivially pluggable, TAEB's name (Tactical Amulet Extraction Bot) has actually never been accurate. But it was such a great name, inciting laughter in anyone we told who was at least as geeky as we are, so we naturally wouldn't change it. TAEB was really a bot framework which happened to ship with a bunch of different AIs. We also (partly as a joke, but partly as a debugging tool) included a <a href="https://github.com/sartak/TAEB/blob/master/lib/TAEB/AI/Human.pm">Human AI</a> that just prompted the terminal what to do every turn. I was fond of labeling TAEB "a framework for programmatic NetHack". Nowadays I'd just shorten that to "a NetHack API", but I was a little more pretentious back then. <h5><tt>TAEB::AI::Interhack</tt></h5> That "programmatic NetHack" label hints at the fundamental design difference between TAEB and Interhack. Because TAEB asserts complete control over the interaction with NetHack (exposing a proper API instead of proxying a practically opaque stream of characters representing NetHack commands), we could go so much further with new interface features that were only a dream in Interhack. For example, autoexplore could never work in Interhack - its conception of the map was about as good as your terminal's (i.e. just a 2d array of colored characters) - but autoexplore would be trivial in TAEB. I even had a pretty good design for an autoplay feature: just use one of TAEB's AI modules until a sufficiently challenging situation came up, then ask the human what to do. Beyond that the human could advise the bot on strategical issues (head to Minetown first, then Sokoban, then Mine's End) instead of tactical or trifling issues (like remembering to pick up food and eating, fighting weak monsters, and navigating across levels to a particular landmark like the nearest altar). In addition to autoexplore and autoplay we could have plugins like: remember which tiles you've ever stepped on for finding the vibrating square and astral portals and even just evading hidden traps, perfect memory of where every item on the ground is (even if your character has been stricken with amnesia), estimating the risk of monsters to determine if they can be windshielded or if they need to be taken out with more caution using magic; ranged weapons; kiting; etc, and even little things like an automap that doesn't suck - it would just know which level you're on and fill in the unexplored sections of the map for you. I wanted someone else to build that Interhack on top of TAEB so that I could continue working on the TAEB bits and contribute the occasional patch to the new Interhack. But no one was ever up for the task. It could very well have been that TAEB was <a href="http://taeb.sartak.org/future.html">not complete enough</a> (which absolutely does matter when it asserts complete control over the interaction with NetHack) to embark on that project. But I know now that if I had just started working on a cool useful project, contributors would have come and eventually someone else could have taken over. Actually, now that I think of it, TAEB gained far more contributors than any other project I've ever started. We've even given a <a href="http://yapc10.org/yn2009/talk/2148">few</a> <a href="http://www.perloasis.info/opw2010/talk/2476">talks</a> about <a href="https://github.com/sartak/TAEB/blob/master/lib/TAEB/AI/YAPC.pm">it</a> at Perl conferences. In all it was a great time and I'm proud of what we accomplished. But I no longer have any desire to break NetHack. Hell I barely even play it any more (much less any other roguelike). Maybe if there's a new release I'll come back just to noodle with all the new stuff. In the meantime I'll continue working on my newer less programmatic hobbies. 2011-01-24 Replace a Lightweight Git Tag with an Annotated Tag http://sartak.org/2011/01/replace-a-lightweight-git-tag-with-an-annotated-tag.html When I released <a href="http://p3rl.org/Jifty::DBI">Jifty-DBI</a> versions <a href="https://github.com/bestpractical/jifty-dbi/tree/0.63">0.63</a> and <a href="https://github.com/bestpractical/jifty-dbi/tree/0.64">0.64</a>, I used a lightweight tag (a simple pointer to a commit) instead of an annotated tag (nearly the equivalent of a commit) to tag the release. At work, we're moving toward using annotated tags for our releases so that they are cryptographically signed, contain author and date, and perhaps most importantly, work better with <a href="http://www.kernel.org/pub/software/scm/git/docs/git-describe.html"><tt>git describe</tt></a>. I was asked to fix the lightweight tags I made for Jifty-DBI releases, which involved a few git incantations I had never used before; hence this post. The basic procedure is: <ol> <li>check out the lightweight tag</li> <li>delete it locally</li> <li>create the annotated commit (backdated to the commit it points to)</li> <li>delete the remote lightweight tag</li> <li>quickly push the annotated tag</li> </ol> Here's the syntax for each of these commands to upgrade tag 0.64 from lightweight to annotated. <pre class="sh code_snippet"> git checkout 0.64 git tag -d 0.64 GIT_COMMITTER_DATE="$(git show --format=%aD | head -1)" git tag -s 0.64 -m '0.64' git push origin :refs/tags/0.64 git push --tags </pre> 2011-01-11 Installing Imager::File::GIF and Imager::File::JPEG with homebrew http://sartak.org/2010/12/installing-imager-file-gif-and-imager-file-jpeg-with-homebrew.html More for my future reference than anything else :) <pre class="sh code_snippet"> brew install giflib brew install libjpeg cpanm --look Imager::File::GIF perl Makefile.PL --libpath=~/.brew/lib --incpath=~/.brew/include make test install cpanm --look Imager::File::JPEG perl Makefile.PL --libpath=~/.brew/lib --incpath=~/.brew/include make test install </pre> 2010-12-27 頑張らなきゃ！ http://sartak.org/2010/09/é ‘å¼µã‚‰ãªãã‚ƒï¼.html <a href="/2010/09/piggybacking-motivation.html">(this but in english)</a> 僕には借りたwiiを返したから、とうとうスーパーマリオギャラクシー2をやってります。スーパーマリオギャラクシーが大好きですが続編の方が楽しいと聞きますね。しかし！日本語が勉強するのに、英語版を買いました。たぶん原作を買えました。でも、アメリカのwiiで出来ないが日本のwiiが必要があることは面倒で高いですね。しかも、マリオのゲームは文書が少ししかないから、やることは勉強ではありませんね。他方、ロールプレイングゲームは… <img src="http://sartak.org/misc/blog/discipline/ps4.png"> 勉強すべきだから、スーパーマリオギャラクシー2をしてる時はちょっと残念です。ある考えが思い浮かんだと思います。それぞれのスターを集めることは、一字の漢字を習ったら…？多いスターがあるから、多い漢字を習うでしょうね。確かに全てのスターを集めようとします！ 規律正しい生活を出来るかな！ ちなみに、このブログを書いてた時は、たくさん辞書を引いたので、たぶん分かりにくいです。まだ頑張らなきゃ！ 2010-09-12 Piggybacking Motivation http://sartak.org/2010/09/piggybacking-motivation.html <a href="/2010/09/頑張らなきゃ！.html">(これを訳しました)</a> I'm finally playing Super Mario Galaxy 2 after having been returned my long overdue Wii. I loved the first game to bits and from everything I've heard, this sequel is somehow even better. But! it's in English and I'm learning Japanese. I could have gotten the Japanese version, but the Wii is region-locked so I'd have to get a Japanese Wii and that is a lot of hassle. Plus, the game isn't exactly chock-full of text, so it's kind of cheating to call playing this studying. A JRPG, on the other hand... <img src="http://sartak.org/misc/blog/discipline/ps4.png"> Anyway, so I feel (only a little) bad about playing Galaxy 2 instead of studying. But maybe I can harness my massive desire to play the game by forcing myself to study before playing more. So, what if I demand learning a new kanji (<a href="http://en.wikipedia.org/wiki/Remembering_the_Kanji">the method for which</a> I still need to write about) for each star? This would result in lots of kanji learned, since the game has lots of stars. It's also very likely that I will get every star because Mario games are goddamn brilliant. The only question is: do I have the discipline to pull this off? Let's find out... 2010-09-11 Super Mario World http://sartak.org/2010/08/super-mario-world.html I was curious how the Japanese version of Super Mario World was different from what I grew up with, so I played it a little bit. Turns out it's less different than I had expected. Given an in-game screenshot, there's nothing that jumps out at me as being different. <img src="http://sartak.org/misc/blog/mario/ghosts.png" alt="Ordinary screenshot of Super Mario World" /> I expected at least one or both of the two words on the screen to have been written as マリオ or 秒/タイム. When you're on the world map, the level names are in Japanese. <img src="http://sartak.org/misc/blog/mario/soda-lake.png" alt="Super Mario World world map" /> As is most of the rest of the in-game text, especially hint boxes, dialog, and narration. <img src="http://sartak.org/misc/blog/mario/castle.png" alt="Mario destroying a castle" /> Remember the Special Zone? You had to unlock it by finding the secret exit in the last level of the already-secret Star World. Your reward for beating this (difficult-to-my-8-year-old-self) Special Zone is a set of graphical changes: a sprite swap of many of the enemies and a slightly different world map. <img src="http://sartak.org/misc/blog/mario/special-zone.png" alt="The Special Zone of SMW" /> In English, these levels' names were taken from surfer slang. <ol> <li>Gnarly</li> <li>Tubular</li> <li>Way Cool</li> <li>Awesome</li> <li>Groovy</li> <li>Mondo</li> <li>Outrageous</li> <li>Funky</li> </ol> I figured I'd play through and see what they originally were. Turns out there are only four names for the eight levels. The first and second level share a name, as do the third and fourth, etc. <ol> <li>おたのしみ　コース (Fun Level) </li> <li>マリオスタッフもビックリ　コース (Mario Staff Also Surprised Level) </li> <li>スペシャリストのためのコース (Level for Specialists) </li> <li>チャンピオンシップのコース (Level for the Championship) </li> </ol> At the very end of Funky (the last Special Zone stage), you are rewarded with 'YOU ARE A SUPER PLAYER!!' written with coins. I was hoping earning the championship would have netted me a スーパープレーヤだよ！！ or something. But instead it says 'YOU ARE SUPER PLAYER!!' with the 'A' article very much missing. <img src="http://sartak.org/misc/blog/mario/super-player.png" alt="You are super player!!" /> People I meet sometimes ask how I learned Japanese (yeah, past tense, what?). This blog post forms part of that answer. I'm learning by having fun. :) 2010-08-22 Learning Japanese with Sentences http://sartak.org/2010/04/learning-japanese-with-sentences.html <h2>Sentences</h2> One of the major components of my Japanese study is sentences. I add sentences written in Japanese to my <a href="http://blog.sartak.org/2010/01/on-learning.html">spaced repetition program</a>, <a href="http://ichi2.net/anki/">Anki</a>. When I review a sentence card, my job is to read the sentence aloud (to confirm that I know the readings for each kanji) and to understand the sentence. <a class="footnote_link" href="#footnote_1" name="footnote_back_1">[1]</a> For simpler sentences which I know well, usually this is automatic and immediate. For new or complicated sentences, understanding sometimes demands translation into English. As I get more exposure to the language, my brain will get better at skipping that slow and lossy translation step. That's the plan anyway! <img src="http://sartak.org/misc/blog/sentences/gehennom.png" alt="Reviewing a sentence" /> Reviewing a sentence card where the answer and translation have both been revealed. <a href="http://www.jnethack.org/">jNetHack</a> is a great source of sentences for me because I know NetHack so well. I always translate from Japanese into English. Never the other way around. For one, the <a href="http://en.wikipedia.org/wiki/Comprehensible_input">comprehensible input hypothesis</a>, which is my guiding light for language study, suggests that production is only a side-effect of language ability, which is developed only when input is received. The input hypothesis even goes so far as to suggest that premature production is harmful, because in doing so you are probably reinforcing incorrect usage. <a class="footnote_link" href="#footnote_2" name="footnote_back_2">[2]</a> Also, more pragmatically, because I am not a native Japanese speaker, or even close to fluent, it is difficult for me to judge whether two sentences have the same meaning, or even similar nuance. So I would have trouble grading myself. This problem doesn't occur when I am producing English, which I have a bit more experience with. For Japanese, I rely only upon correct sentences written by native speakers, instead of whatever nonsense I come up with. :) I also avoid studying soulless <a class="footnote_link" href="#footnote_3" name="footnote_back_3">[3]</a> vocabulary lists. Briefly, consider the difference between a lethal injection and a mortal injection. Because you've never heard of a mortal injection before, <a class="footnote_link" href="#footnote_4" name="footnote_back_4">[4]</a> it sounds wrong, even though lethal and mortal both mean deadly. If I want to learn a new word, I force myself to find a complete sentence which uses it. So where am I getting these sentences? It turns out the answer is "from all over the place." But before I can talk about that, I want to explain a bit more about Anki. <h2>Facts vs Cards</h2> Anki is great for separating facts from cards. A card is familiar; each has a front (the question) and a back (the answer). A fact, on the other hand, is a set of key/value pairs. You create a fact then generate one or more cards from it. For example, if you are studying world nations, you could have a fact for each country which includes Country Name, Capital, Languages, etc. Then you could generate many cards which test you on going from country to capital, country to language, capital to country, etc. Anki's pleasant design turns out to be a great timesaver since it frees you from repeating yourself. You only type the country's name once but it can be used in many cards. <img src="http://sartak.org/misc/blog/sentences/nation.png" alt="Creating a new fact in Anki" /> This is fact creation screen, which you will see a lot. <img src="http://sartak.org/misc/blog/sentences/generate.png" alt="Generating cards from facts in Anki" /> This is the card template screen, which you will rarely look at. <a class="footnote_link" href="#footnote_5" name="footnote_back_5">[5]</a> We added only four facts, but from them we created twenty-eight cards, each of which tests a single piece of data. This way, your "language of Australia" card will be independently spaced from your "capital of Australia", which is good when you have trouble with one of them. If you already know "English" but "Canberra" just won't stick in your mind, that's okay. You won't have to review that the Australians use English nearly as often as you review their capital city. <h2>Sources</h2> One of the fields that my sentence facts have is 'Source'. Other than the sentence itself, Source is the only field that I require myself to fill in. Translation, readings, context, etc. are optional. Knowing where each sentence came from is useful because it lets me gauge the trustworthiness of each sentence. I always trust the correctness of sentences uttered in, say, film, whereas an offhand remark on Twitter could easily have a typo or unusually playful grammar. It's also useful to know whether I have accidentally incorporated feminine language into my lexicon. My diligence in citing every sentence I learn also lets me geek out and analyze precisely how I am learning Japanese. I wrote a script to pull all the values for the Source field out of the database and categorize them according to rules like: <pre class="perl code_snippet"> qr{The Matrix} => { speech, native, movie, source("The Matrix") }, qr{iPod USB/dock cable box} => { text, native, medium => 'misc', source("Miscellaneous"), }, qr{IRC PM with (\w+)} => { student, correspondence, author(defer { $1 }), }, qr{mt.endeworks.jp} => { blog('lestrrat') }, qr{Conversation at Ebisuya} => { convo, native }, </pre> <h2>My Input</h2> Another component of my study is immersion. I try to always have Japanese music in my environment. I can't escape the deluge of Japanese tweets from those I follow. There's also movies, books, IRC, and when I'm lucky even in-person conversation. I expose myself to as much Japanese-meant-for-native-speakers as possible. Every now and then I get a sentence I almost understand, except for a word or a reading. For example, as for the sentence in the marked-up image at the top of this post, I already knew and understood everything except the reading for 門 (gate) which is もん (mon). So for each so-called i+1 sentence <a class="footnote_link" href="#footnote_6" name="footnote_back_6">[6]</a> , I look up the one piece I'm missing then add it to Anki. Repeat until fluent. But this, for sanity's sake, is tempered with Japanese-as-a-second-language materials. It would be frustrating to listen to and read complete gibberish all day, so I begrudingly use a <a href="http://www.guidetojapanese.org/learn/grammar">grammar guide</a>. I also use various other second-language materials <a class="footnote_link" href="#footnote_7" name="footnote_back_7">[7]</a> , as long as they meet my impossibly high standards. <a class="footnote_link" href="#footnote_8" name="footnote_back_8">[8]</a> Turns out my setences are almost exactly evenly split between these two types. Eventually the number of sentences which are intended for native speakers will dwarf the sentences intended for students. I seem to be on the right track, as evidenced by this shiny chart <a class="footnote_link" href="#footnote_9" name="footnote_back_9">[9]</a> (the x-axis is of course time but the unit is unimportant). <img src="http://sartak.org/misc/blog/sentences/audience.png" alt="A chart of the intended audience by week" /> It's also interesting to look at the ratio of added sentences which I have read (94%) versus sentences I have heard (6%). <img src="http://sartak.org/misc/blog/sentences/media-type.png" alt="A chart of sentences I have read versus those I have heard" /> There are a few reasons for this disparity. I am reluctant to add sentences from music. Though lyrics are usually grammatically correct, it's fair to say they probably utilize some iffy language in order to fit the struture of the song. Also there's the peculiarity of kanji and <a href="http://en.wikipedia.org/wiki/Remembering_the_kanji">how I'm studying them</a>. I do much better with words written in kanji than those nd usually not a lot of context is needed (as opposed to a sentence in the middle of a book). The games from which I've mined most are Phantasy Star 4 and jNetHack (which I beat yesterday!). I'm not really into anime, but the sentences I have are from Fist of the North Star: <object width="480" height="385"><param name="movie" value="http://www.youtube.com/v/_N2pPrIxboI&hl=ja_JP&fs=1&rel=0"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://www.youtube.com/v/_N2pPrIxboI&hl=ja_JP&fs=1&rel=0" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="480" height="385"></embed></object> It's also worth noting that knowing the entire English script of The Matrix has been helpful for watching and learning from the Japanese dub. Just sayin'. <img src="http://sartak.org/misc/blog/sentences/matrix-pie.png" alt="Truisms" /> I got this idea of learning from sentences using spaced repetition from the guys at <a href="http://www.antimoon.com/">Antimoon</a>. They learned English to native levels while living in Poland. In a year, I will rerun the script I used to generate these charts to see how they compare. In the meantime, I better get to back to <a href="http://twitter.com/sartak/status/12501182382">studying</a>! <h5>Footnotes</h5> <hr /> <ol class="footnotes"> <li class="footnote"> <a name="footnote_1"></a> I too-often have to mark sentence cards incorrect for flubbing a reading even though I know what the sentence means. So I am sorely tempted to generate two separate cards for each sentence, one which tests readings and one which tests understanding. It would be easy to do, but I'm not entirely convinced that I should. <a href="#footnote_back_1" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_2"></a> Though, of course, it's really fun to converse with speakers of your second language. The whole point of language is to communicate after all. So production there is fine. You just should not do it during review. <a href="#footnote_back_2" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_3"></a> I've been using this word a lot lately. It's excellent. <a href="#footnote_back_3" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_4"></a> Except when listening to language blowhards like me explain this point. <a href="#footnote_back_4" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_5"></a> If you're familiar with <a href="http://moose.perl.org">Moose</a>, then maybe a useful analogy is that the Add Facts screen is like <code>use Moose</code>, but the Card Template screen is like delving into <code>Class::MOP</code>. <a href="#footnote_back_5" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_6"></a> Here, i is everything I understood about Japanese before adding the Gehennom sentence. +1 represents the reading of gate that I learned through SRSing this sentence. Now my i is a little bit higher, and I'm a little bit more prepared for the next sentence. <a href="#footnote_back_6" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_7"></a> Such as various Japanese-English dictionaries. I haven't started using a monolingual dictionary yet. That is a scary - but necessary - step to take. Very soon! <a href="#footnote_back_7" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_8"></a> I avoid learning sentences written or spoken by fellow learners - even those who are far better than me with the language. Sorry! Nothing personal, but my goal is to eventually confuse people into thinking my first language is Japanese. :) <a href="#footnote_back_8" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_9"></a> The charts in this post were generated with <a href="http://p3rl.org/Chart::Clicker">Chart::Clicker</a>, written by the inimitable <a href="http://www.onemogin.com/">Cory Watson</a>. Any anti-Tuftean errors are mine, not the module's. <a href="#footnote_back_9" class="footnote_link_top">↩</a> </li> </ol> 2010-04-20 On Learning http://sartak.org/2010/01/on-learning.html At a party last night, I explained spaced repetition and Anki to two separate groups of people. <a class="footnote_link" href="#footnote_1" name="footnote_back_1">[1]</a> I'm really enthusiastic about Anki so I'm shooting for as many converts as I can in a twenty-four hour period. <h2>Spaced Repetition</h2> <a href="http://en.wikipedia.org/wiki/Spaced_repetition">Spaced repetition</a> is the principle that the best time to review a fact is just before you forget it. If spaced repetition weren't so effective, I bet the advertising industry would be far smaller. Try completing these slogans: <ul> <li>I can't believe it's not butter </li> <li>Melts in your mouth, not in your hand </li> <li>Just do it </li> <li>The breakfast of champions </li> </ul> You haven't explicitly memorized these slogans, but I bet you'd be hard-pressed to forget them. (Most of?) you know them effortlessly because you see and hear them every now and then. The same principle underlies spaced repetition. A spaced repetition system (SRS) is basically a program that tests you using flash cards. The major departure from an ordinary flash card program is that an SRS has a scheduler which decides when to test you on each fact. The scheduling of a fact is influenced by how well you know that fact. That's the essence of an SRS; many SRS programs have all sorts of extra functionality on top of that since they're used (and developed) by people who are humorlessly serious about learning. Like me. I really like spaced repetition because it is rarely boring. You are never bogged down reviewing the same dozen cards that you know. With traditional flash cards, facts you know very well and facts you don't have the same precedence. Spaced repetition software tests you on facts you know well far less often than facts you don't know well yet. Unlike people, not all facts are created equal. You don't need to review every day that Boston is the capital of Massachusetts, especially if that's where you live. On the other hand, you may need to be reminded often that Regina is the capital of Saskatchewan. I sure do. Who even lives there. Ready for another example? The kanji 雑 (miscellaneous) is pretty new to me, so I'm likely to see it again in the next day or two. On the other hand, every time I've been tested on the kanji 副 (vice-) I've gotten it correct, so I'm not going to see it again for four months. And I bet I will remember it come May. If not, Anki will adjust the card's interval accordingly. As of this writing I have 960 cards in my deck. It would take hours to review all of them every night. Not to mention that such reviews would be boring as hell, since I know most of the facts very well. <a class="footnote_link" href="#footnote_2" name="footnote_back_2">[2]</a> Thanks to spaced repetition knowing that I am capable of learning, I review only about 64 cards per day. That may still seem like a lot, but such a review takes only fifteen or twenty minutes a day. I'll gladly review 64 distinct cards a day over doing hundreds (thousands?) of repetitions of the same set of flash cards the night before an exam. Furthermore, SRS is designed to build memories that last longer than a semester. Most spaced repetition systems will actually let you grade yourself based on how well you know the fact. There's obviously a large gap between struggling to recall a fact before eventually getting it and having the answer pop into your head before you even finish reading the question. You will see cards of the former type more often than cards of the latter type. <h2>Anki</h2> <a href="http://ichi2.net/anki/">Anki</a> is one of the best spaced repetition systems. I use it to learn Japanese. I trust Anki so much that I feel if my deck lacks a particular word, phrase, kanji, or proper name, then I do not know it. I have recently begun to use Anki for more than just Japanese. I have a Miscellaneous deck into which I put whatever I feel is worth knowing. I learn state capitals, Canadian province capitals <a class="footnote_link" href="#footnote_3" name="footnote_back_3">[3]</a> , and words that I am not comfortable using in a sentence or whose definitions are vague. I should also start putting in different kinds of things like maps with one country's label erased and Perl's more unusual special variables. Maybe I'll even prepare for the CPAN drinking game. :) Getting more exotic, there are people who put <a href="http://supermemoadventures.blogspot.com/2009/12/art.html">art</a> into the SRS to learn more about art history, and even quotes from <a href="http://www.alljapaneseallthetime.com/blog/why-the-way-we-read-sucks-and-how-to-fix-it-part-4-why-srs-personal-development-books">personal development books</a> so that the ideas are constantly in mind, which is more likely to effect change. I am not in school any more, but if I were, I would definitely have an Anki deck for each of my classes. I used Anki to learn the Japanese poem <a href="http://en.wikipedia.org/wiki/Iroha">Iroha</a>. I'm not very into poetry, but Iroha is worth knowing for several reasons. It's a pangram of the (circa 1000AD) Japanese syllabary, so it immediately has Cool Points. The order of the kana in Iroha is still used in modern Japan for labeling Go boards and theater seats. I use it to practice writing all the kana. Iroha is far too long to use only one card, so the best way to learn it (as pointed out by <a href="http://study-shack.com/creating-cards-models-for-all-classes">Study Shack</a>) is by using eight cards, one for each line of the poem. Each card has a different line blanked out and your job is to produce the missing line. Here's what Anki's review interface looks like (with the answer revealed in the lower half): <img src="http://sartak.org/misc/blog/anki/iroha.png" /> Anki has a convenient feature called 'cloze deletion' that makes creating eight similar cards like this take only a minute. By the way seeing the other seven lines of the poem in every card is like even more spaced repetition, for free. <h3>Design</h3> Facts are not limited to a front and back. Each fact belongs to a model. The model lets you control which fields are available (including requiredness and uniqueness constraints on each field). Some models I use are Sentence (which has fields 日本語, Meaning, Readings, and Source) and Capital (Region, Capital). Models also control how cards of that model are displayed. <img src="http://sartak.org/misc/blog/anki/model.png" /> As you can see, you can also generate several cards from a single fact. Not only do I want to test myself to be able to produce the capital of a given region, but I also want to test myself on producing a region given just its capital city. You can have as many fields in a model as you want, and you can generate many different types of cards for each fact. You could create eight cards based on ten fields if you were so inclined. This is powerful stuff! Anki lets you uphold the Don't Repeat Yourself principle dear to many programmers. <h3>Interface</h3> Anki has a lot of gears and widgets, but what matters most is the interface you see when you're reviewing a fact. It may not look like it, but Anki's review interface is actually a fully-featured web browser. With Javascript. It has, for example, <code>alert()</code>. Providing a full web browser gives Anki basically limitless flexibility! <img src="http://sartak.org/misc/blog/anki/alert.png" /> Being able to use arbitrary fields in a model works out amazingly well when you have Javascript at your disposal. You can make one of the fields a YouTube video and show it if you want. <a class="footnote_link" href="#footnote_4" name="footnote_back_4">[4]</a> <h3>Plugins</h3> Anki is written in Python. Not only is its UI extensible, but the backend seems to be designed that way as well. There are a dozens of plugins that are easily available in File > Download > Shared Plugin. A lot of them offer incremental improvements, but there are two that have made Anki significantly better for me. <h4>Learn Mode</h4> I've noticed a nasty corner case of spaced repetition as implemented by Anki. If you get a (usually brand new) card wrong, you'll see it again very soon, sometime within the same review. You'll probably then remember it easily thanks to short-term memory (or perhaps very fleeting long-term memory). So you mark it honestly, as a three, which schedules it for review in two or three days. By the time you review it again you'll have forgoten it. Lame. The Learn mode plugin alleviates this problem by performing a review with much shorter intervals. Cards are scheduled only seconds and minutes in the future, rather than d until there are no more cards for review (since they'rel scheduled a few minutes into the future). This usually results in 10 or 15 reviews for each fact. Then I go back to the Anki on your iPod or iPhone. You sync with regular Anki using http over wifi; the plugin provides a web server. <a class="footnote_link" href="#footnote_5" name="footnote_back_5">[5]</a> There's another interface if your iPod/iPhone is jailbroken - Anki Mini - with more features but I had less luck in getting it to work correctly. The iAnki interface is much simpler than Anki's. You can't add or edit cards, or even look at your card list. You can use iAnki only to review. Despite that, iAnki is still worth having. Maybe someday they (or I!) will add lots more functionality. When learning kanji it's really important to practice writing them out. For a while I used the air or the back of my hand. Turns out I'm way too forgiving of errors when I do that. So I really wanted a canvas directly in iAnki that I could use to grade myself properly. I found <a href="http://www.mblondel.org/journal/2008/08/01/web-canvas/">Web Canvas</a> which turned out to be quite good in terms of functionality and ease-of-use. I had to patch it to accept my iPod's touch gestures (which was incorporated upstream) but after that it was pretty smooth sailing. (update: I've also submitted some <a href="http://github.com/sartak/ianki">patches</a> to iAnki itself). Here's what my iAnki looks like for testing a kanji: <img src="http://sartak.org/misc/blog/anki/IMG_0021.PNG" /> This is asking us to produce the kanji for "frost". <img src="http://sartak.org/misc/blog/anki/IMG_0023.PNG" /> If I have trouble remembering the kanji, I can look at the primitive story of the kanji. Stories are a peculiarity of the <a href="http://www.amazon.com/gp/product/0824831659?ie=UTF8&tag=sartak-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0824831659">method I am using to learn kanji</a> which will be the topic of another post. The "(primitives)" link had a javascript onclick handler that replaced itself with the story. Hiding the story behind a link is handy because I usually do not want to see the story unless I am stuck. <a class="footnote_link" href="#footnote_6" name="footnote_back_6">[6]</a> <img src="http://sartak.org/misc/blog/anki/IMG_0024.PNG" /> Now I've written out the kanji. As fun as touchscreens are, it's really hard to demonstrate good penmanship with them. This is actually embarrassingly bad. <a class="footnote_link" href="#footnote_7" name="footnote_back_7">[7]</a> <img src="http://sartak.org/misc/blog/anki/IMG_0025.PNG" /> Finally I check the answer against what I wrote. Since I got this correct, and I recalled it effortlessly, I marked it as a four. Anki scheduled this kanji to appear again in about eight weeks. <h3>Graphs</h3> Most importantly, Anki lets you measure your worth with graphs and statistics! This is a graph of facts I've added over the past month. The spike on the fifth is when I played Phantasy Star IV <a class="footnote_link" href="#footnote_8" name="footnote_back_8">[8]</a> all day and added a bunch of sentences. <img src="http://sartak.org/misc/blog/anki/graph.png" /> It also tells you how difficult each card is. So here are the cards that trouble me most. I should give them special treatment, such as cramming them for a while or something. <img src="http://sartak.org/misc/blog/anki/ease.png" /> All of your cards are stored in a SQLite database so if Anki doesn't offer exactly the kind of information you want, you can still get at it. I used the database directly to produce a <a href="http://sartak.org/misc/kanji.html">calendar of kanji</a> I've learned. That was a little worrying; I didn't realize how few kanji I learned in November and December (compared to October). But for the same reason it was also motivating. <hr /> Anki is a wonderful program. I will use Anki and its successors for the rest of my life. It's almost too bad Anki is free software.. I would have liked to be able to retire on commission. ;) <a class="footnote_link" href="#footnote_9" name="footnote_back_9">[9]</a> If I've sold you on SRS then download <a href="http://ichi2.net/anki/">Anki</a> then head on over to the <a href="http://study-shack.com/table-of-contents">Study Shack</a> which talks about what kind of cards are good, and other SRS management tips. <hr /> <h5>Footnotes</h5> <ol class="footnotes"> <li class="footnote"> <a name="footnote_1"></a> Yes I know I attend wild parties. :) <a href="#footnote_back_1" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_2"></a> Similarly, writing this is boring as hell because I know all of this. But you don't. So you're welcome. As if you were even grateful to begin with... Just get out my face and click this: <a href="#footnote_back_2" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_3"></a> Do you remember the capital of Saskatchewan? It's still Regina <a href="#footnote_back_3" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_4"></a> I don't know. That's a useless example. I have a better one later. It lives between where you came from and where you are now. <a href="#footnote_back_4" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_5"></a> The web server is WSGI-based. Also the Javascript bits of iAnki use Joose. We have these things in Perl F.Y.I.! <a href="#footnote_back_5" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_6"></a> This is the better example I was talking about. Case closed. <a href="#footnote_back_6" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_7"></a> In ten years I'm going to happen upon this image and cry. <a href="#footnote_back_7" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_8"></a> which owns <a class="footnote_link" href="#footnote_10" name="footnote_back_10">[10]</a> <a href="#footnote_back_8" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_9"></a> Quick! Capital of Saskatchewan! Go! Don't look at footnote 3 either. <a href="#footnote_back_9" class="footnote_link_top">↩</a> </li> <li class="footnote"> <a name="footnote_10"></a> Sorry about all the footnotes. I made it really easy to make them. :) <a href="#footnote_back_10" class="footnote_link_top">↩</a> </li> </ol> 2010-01-01 Jifty's Request Inspector http://sartak.org/2009/10/jifty-s-request-inspector.html <a href="http://jifty.org">Jifty</a>'s <a href="http://p3rl.org/Jifty::Plugin::RequestInspector">RequestInspector plugin</a> provides you with debugging information for HTTP requests. You can examine the SQL queries that were issued in servicing the request, inspect a <a href="http://p3rl.org/Devel::NYTProf">Devel::NYTProf</a> profile of the request, and more. The information is presented in an (admittedly ugly) admin panel. Here's what our <a href="https://github.com/bestpractical/App-Changeloggr">Changelogger</a>'s request inspector page looks like: <img src="http://sartak.org/misc/blog/requestinspector-basic.png" /> If we click into one of the requests, we can get an overview of that request. <img src="http://sartak.org/misc/blog/requestinspector-request.png" /> We can inspect in detail a particular part of a request with another click. <img src="http://sartak.org/misc/blog/requestinspector-dumpreqres.png" /> <h2>Backend</h2> The frontend is good enough. It gets the job done. But what I am really interested in, and what I had far more fun creating, is the backend that powers this. RequestInspector is a Jifty plugin. Its job is to manage all of the other plugins that inspect requests (such as <a href="http://p3rl.org/Jifty::Plugin::SQLQueries">SQLQueries</a>, <a href="http://p3rl.org/Jifty::Plugin::DumpRequestResponse">DumpRequestResponse</a>, <a href="http://p3rl.org/Jifty::Plugin::NYTProf">NYTProf</a>, and <a href="http://p3rl.org/Jifty::Plugin::Gladiator">Gladiator</a>). RequestInspector informs the plugins of interesting events such as: a new request is beginning, that request finished. Each plugin produces various kinds of data. That data is completely opaque to RequestInspector. The data can be an object, a number, a code reference, or whatever else. All RequestInspector does is keep track of that data, handing the data to the plugin when it needs to do something with the data. When a new request begins, RequestInspector calls the <tt>inspect_before_request</tt> method on each plugin. When the request ends, <tt>inspect_after_request</tt> is called on each plugin. The interesting bit is that the opaque data is threaded between the return values and parameters of these methods. The return value of <tt>inspect_before_request</tt> is passed to that plugin's <tt>inspect_after_request</tt>. <tt>inspect_after_request</tt>'s return value is stored for later rendering. (In what amounts to a global variable — sorry!) When the user wants to see a request, RequestInspector calls the <tt>inspect_render_summary</tt> method on each plugin. When the user wants to see the detailed analysis by a particular plugin for a particular request, RequestInspector calls the <tt>inspect_render_analysis</tt> method on that plugin. It is completely up to the plugin to decide what to render. Both methods receive the data it returned from <tt>inspect_after_request</tt> for that request. <h3>Benefits</h3> You might wonder why this design is better than, say, a specific data format to which each plugin must adhere. It is unclear what such a data format would look like. I don't think a single data format could usefully encompass the different kinds of data that these plugins produce. While SQLQueries's data is complex data structure with a lot of information, NYTProf's is basically just a random number used to track the on-disk profile directory. The format would certainly evolve over time. This means that plugins would have to change as well just to keep up with RequestInspector. With my design, plugins really only need to change when they want to support newer RequestInspector features. It's simpler for plugins to let them dictate how they want to store whatever they are tracking. It's simpler for the RequestInspector core too, since it is free from having to examine the plugins' data. Finally, the RequestInspector plugin can get arbitrarily complex and the plugins do not have to care about it. The RequestInspector already lets you filter out URLs you know you will not want to inspect. The individual request inspector plugins do not have to implement that functionality. In the future, I plan to add a feature to RequestInspector that lets you see which requests took the most amount of database time, or leaked the most amount of memory, etc. All the feature will require of plugins is that they implement an optional <tt>inspect_compare_requests</tt> method. The method will receive that plugin's data from two requests and returns -1, 0, or 1. So trivial that I should just do it now.. :) Basically, RequestInspector leverages the <a href="http://en.wikipedia.org/wiki/Principle_of_least_knowledge">Principle of Least Knowledge</a> to keep things simple, easy, and flexible. Shriram Krishnamurthi's <a href="http://www.international-lisp-conference.org/2009/speakers#krishnamurthi_shriram">Moby Scheme Compiler</a> talk was particularly influential on this design. He described a similar (though simpler) system that he uses to make programming fun and interesting for middle school kids. 2009-10-07 Google Wave Mania http://sartak.org/2009/09/google-wave-mania.html I've been on Google Wave since June 11th, thanks to a friend at Google. I haven't been using the account much due to other pursuits in my life. But tonight, since they're rolling to production (I'm in as sartak@googlewave.com), they gave those in the developer preview invitations. Since I know a lot of people are very excited about Google Wave I thought I would be fair and ask on Twitter... <blockquote>I have google wave invites. DM me if you want one.</blockquote> Almost immediately I was flooded with random people who have been scouring Twitter for idiots like me. <img src="http://sartak.org/misc/blog/wave-twitter.png" /> They also followed me on Twitter because that will help their chances I guess?? (edit: miyagawa pointed out that they did this so I could DM them their invitation) <img src="http://sartak.org/misc/blog/wave-gmail.png" /> Of course, friends who saw my tweet pestered me elsewhere too.. <img src="http://sartak.org/misc/blog/wave-irc.png" /> I lacked the foresight to turn off the DM-to-cell-phone Twitter notifications. <img src="http://sartak.org/misc/blog/wave-phone.png" /> Dayum... Google really knows how to generate interest. Update: Oh god it will never end <img src="http://sartak.org/misc/blog/wave-jshirley.png" /> 2009-09-30 YAPC::Asia 2009 Moose Course http://sartak.org/2009/09/yapc-asia-2009-moose-course.html <a href="http://www.flickr.com/photos/sartak/3921128445/" title="Moose course by sartak, on Flickr"><img src="http://farm3.static.flickr.com/2451/3921128445_ffa1471fef.jpg" width="500" height="375" alt="Moose course" /></a> I have received a lot of good feedback about my <a href="http://conferences.yapcasia.org/ya2009/">YAPC::Asia 2009</a> <a href="http://moose.perl.org">Moose</a> <a href="http://conferences.yapcasia.org/ya2009/talk/2192">course</a>. Several people on IRC and Twitter <a href="http://twitter.com/lesamoureuses/statuses/4029397889">thanked me</a>. I got second-hand reports of students taking the course then immediately being able to write useful Moose code. That is nothing short of ideal. It makes me very happy that people not only enjoyed the course, but that it was practical. Several people also wrote blog posts about the course's content and my presentation. <ul> <li>lesamoureuses's <a href="http://d.hatena.ne.jp/lesamoureuses/20090915/1253023402">YAPC::Asia2009の特別研修「Moose入門、モダーンなオブジェクト指向システ>ム」が超良かった！</a></li> <li>ziguzagu's <a href="http://ziguzagu.org/2009/09/yapcasia-2009--.html">YAPC::Asia 2009 特別研修 - Moose 入門</a></li> <li>Gardejo's <a href="http://blog.eorzea.asia/2009/09/post_65.html">Mooseの中の人によるMoose研修が素晴らしすぎる (YAPC::Asia 2009)</a></li> </ul> As expected, the students loved the exercises. I feel very strongly that the exercises are the best part of the course. They force you to make sure you understand what you have just learned. It's handy to have a Moose man there to help students when they get stuck. My only regret about this iteration of the course is that we had to skip the last two exercises for time constraints. To the students' credit, every single one did stay well over an hour past the scheduled end time. I was pleasantly surprised that I received such nice remarks about my live coding. As these are not my slides, I sometimes had to dive into vim to explain a tangential point about Moose. For example, demonstrating the use of <tt>has</tt> after <tt>__PACKAGE__->meta->make_immutable</tt> (which led to <a href="http://rt.cpan.org/Public/Bug/Display.html?id=49680">a bug report</a> and then <a href="http://git.shadowcat.co.uk/gitweb/gitweb.cgi?p=gitmo/Class-MOP.git;a=commitdiff;h=e347ce3fe9e40dc060e73e07f282729f4191c878">a bug fix</a> — all during the course!). I very much enjoy this style of presentation, but it is difficult to come up with real examples on the fly. It feels great when it all just works. I had a whole lot of help in making this course happen. A very big thanks to Daisuke Maki and the <a href="http://japan.perlassociation.org/jpa">Japan Perl Association</a> for establishing and marketing the course (and of course running this excellent YAPC::Asia!). Thanks to Emerson Mills for working with me to make the course happen, and for translating the exercises. Thanks to Nathaniel for interpreting my presentation. I did not make it easy for him! Thanks to Kenichi Ishigaki (charsbar++) for helping out with translation, for buying (with his own money) my throat lots of tea, and for being generally awesome. Thanks to all of the students for asking very good, deep questions. It's very satisfying to know one's audience is attentive and thoughtful. Finally, thanks to Dave Rolsky for letting me use his <a href="http://git.shadowcat.co.uk/gitweb/gitweb.cgi?p=gitmo/moose-presentations.git;a=tree;f=moose-class">course material</a> yet again. Next week he at long last gets to <a href="http://blog.urth.org/2009/09/moose-class-in-minneapolis-september-23-2009.html">teach his own course</a> for the first time in Minneapolis. I look forward to giving this course again. I really enjoy spreading the Moose love. 2009-09-17 Reflections on YAPC::NA 2009 http://sartak.org/2009/06/reflections-on-yapc-na-2009.html I had a lot of fun at <a href="http://yapc10.org/yn2009/">YAPC::NA</a> this year. I met some <a href="http://pbfcomics.com/?cid=PBF152-Scorpy_the_Forest_Friend.gif">forest friends</a> that didn't make it last year, such as <a href="http://ilmari.org/">ilmari</a>, <a href="http://clownpenis.fart/">Nick Perez</a>, and <a href="http://dylan.hardison.net">Dylan Hardison</a>. I also laughed until it hurt. A few times. I was a hermit on Sunday and Monday nights because I had not finished my slides. That was very unfortunate, so I will have future talks ready well in advance. Jitting slides is so not my style. <h3>Moose Course</h3> The eight hour Moose course went very well. I was a passable <a href="http://blog.urth.org/2009/06/i-wont-be-at-yapc-10.html">Rolsky-substitute</a>. The <a href="http://blog.afoolishmanifesto.com/archives/844">feedback</a> I recieved was (nearly) unanimously good, so I'm thrilled that people enjoyed it. There were some minor complaints, so if I ever teach a course again I'll be sure to keep them in mind. This year, YAPC::NA is running surveys on talks and courses, so I expect I'll learn more ways that the class could be improved. If you attended the class, please do not worry about offending Dave or me if you disliked something. I was under-prepared for the class, but <a href="http://git.shadowcat.co.uk/gitweb/gitweb.cgi?p=gitmo/moose-presentations.git;a=tree;f=moose-class">the course material</a> was very good. I got the impression that the exercises were extremely helpful. In particular, having a good test suite (powered by the metaobject protocol!) was very useful. The exercises were also useful for me because they let me sit down. Speaking on your feet for so many hours is tough. Like the first version of anything, the course material contained some typos, inconsistencies, and bugs. It was pleasant to fix these and push them back to the source repository while people worked on the exercises. For some reason, we could not get public (i.e. noncommitter) access to the repository working, perhaps due to DNS issues. It would have been useful to have people pull my changes to fix the exercises. Instead, I settled for announcing what the problems were and how to fix them. <h4 id="context">Context</h4> Early in the course I had a vitriolic rant about Perl's notion of context. On the whole I think context is a nice solution to a certain class of problems, but it definitely has some edge cases where it really bites programmers. For example, propagating context is very fiddly. You must propagate context if you wrap a function to do work after it is called without affecting its return value. <pre class="perl code_snippet"> sub method { my $self = shift; # call SUPER::method in the right context. # not handling void context is a BUG! my @ret; if (wantarray) { @ret = $self->SUPER::method(@_); } elsif (defined wantarray) { $ret[0] = $self->SUPER::method(@_); } else { $self->SUPER::method(@_); } $self->called_method($self->called_method + 1); # return what the caller asked for. # void context doesn't need to be explicitly handled here # .. or does it? maybe! return @ret if wantarray; return $ret[0]; } </pre> I don't recall mentioning in the class that Moose does all of this for us in the <tt>after</tt> method modifier: <pre class="perl code_snippet"> after method => sub { my $self = shift; $self->called_method($self->called_method + 1); }; </pre> The other painful detail about context is in returning a list in scalar context. Most people know that an array in scalar context evaluates to the number of elements: <pre class="perl code_snippet"> sub users { my $self = shift; my @users = $self->objects('users'); return @users; } my $users = $game->users; # 51 </pre> However, a lot of people don't know that there is an irritating amount of subtlety in evaluating a list in scalar context: <pre class="perl code_snippet"> sub administrators { my $self = shift; return ('Sartak', 'autarch'); } my $admins = $game->administrators; # autarch </pre> Instead of receiving the count of administrators, we get the last element of the list. The problem is actually the comma. Due to C's strong influence on Perl 5, the comma is not just a list separator. In scalar context it acts as a sequencing operator. We're evaluating the string 'Sartak', throwing it away, then evaluating the string 'autarch'. There is no list at all here! <blockquote>Binary "," is the comma operator. In scalar context it evaluates its left argument, throws that value away, then evaluates its right argument and returns that value. This is just like C's comma operator. – <tt><a href="http://perldoc.perl.org/perlop.html#Comma-Operator">perldoc perlop</a></tt></blockquote> Almost everywhere else in the language, arrays and lists are interchangeable. I wish this inconsistency could be excised. Thanks to <tt>do BLOCK</tt>, we do not need C's comma operator at all. <h3>Extending Moose</h3> On Tuesday morning I presented <a href="http://sartak.org/talks/yapc-na-2009/extending-moose/extending-moose.pdf">Extending Moose for Applications</a> <a href="http://sartak.org/talks/yapc-na-2009/extending-moose/extending-moose.key">(keynote source)</a>. There were maybe 20 attendees. I blame the 8AM start time. Oh well. I'm told that at one point someone walked out saying the talk got "too abstract". It certainly is a very abstract topic, but I hope the examples showed that there is value to be had in using the ideas I presented. Some good questions were asked, and some people did get it. I conveyed the point I wanted to make, which is that the metaobject protocol uses concepts you already know well. I encourage you to read the slides. There are notes which is basically what I spoke when I was presenting, so they should be pretty readable. Please let me know how I did. I've received very little feedback on this talk. <h3>TAEB</h3> One of my good friends <a href="http://tozt.net/">Jesse Luehrs (doy)</a> gave a lightning talk about our NetHack bot, <a href="http://taeb.sartak.org/">TAEB</a>. <img src="http://sartak.org/misc/blog/doy-taeb.jpg" /> The line "The code I'm working on isn't really a NetHack bot; it's more of a framework for writing NetHack bots" got quite a few laughs. It's completely true, too. The topic of the talk was writing a new AI for this bot framework. The bitrotted web interface seemed to please people. It's too bad they saw how it was when I first wrote it. Since I took those screenshots, I improved it quite a bit, making it use NetHack's colors and providing an AJAXy interface. Oh well, I should really get that working again. This lightning talk was a hit. Several people cheered! Great job, Jesse. <h3>Others' Talks</h3> My favorite talk, by far, was <a href="http://thelackthereof.org">Brock Wilcox</a>'s <a href="http://thelackthereof.org/CGI_Inspect">CGI::Inspect talk</a>. As I said during the talk, when I wrote <a href="http://p3rl.org/Carp::REPL">Carp::REPL</a>, I was actually aiming for exactly what <a href="http://p3rl.org/CGI::Inspect">CGI::Inspect</a> is. And I meant it! About two years ago, it was mentioned to me how when there's an error in your Django application, it opens an AJAXy REPL for you. I figured out how to do some of that - REPL on error - but I never actually hooked it up to <a href="http://jifty.org">Jifty</a>. Now I don't have to. Bravo, Brock! Another talk I enjoyed was <a href="http://rjbs.manxome.org/">Ricardo Signes</a>'s <a href="http://yapc10.org/yn2009/talk/1942">Git is Easy talk</a>. I knew most of the material presented, but since seeing that talk I have had a lot more confidence in using git. In particular, remotes now make sense to me. I also now use <a href="http://gitx.frim.nl/">GitX</a> to visualize where other developers are relative to me. <a href="http://slowass.net/~scott/">Scott Walter</a>'s <a href="http://yapc10.org/yn2009/talk/1974">Perl in Vegas talk</a> was fascinating. He drew an interesting parallel between the incredible strictures of Vegas slot machines versus America's completely opaque and mostly unregulated voting machines. One strange requirement of the slot machines is that code must be compiled. Though Perl is an interpreted language, it provides a <a href="http://www.perl.com/doc/manual/html/utils/perlcc.html">perlcc</a> utility to "compile" the code. <a href="http://weftsoar.net/~hdp/">Hans Dieter Pearcey</a>'s <a href="http://yapc10.org/yn2009/talk/1986">Dist::Zilla - Automating quality since 2008</a> reminded me of its excellent plugin-centric design: <img src="http://sartak.org/misc/blog/dist-zilla-plugins.png" /> This slide directly caused me to submit a talk about <a href="http://sartak.org/talks/yapc-asia-2009/api-design/abstract.txt">API Design</a> to <a href="http://conferences.yapcasia.org/ya2009/">YAPC::Asia 2009</a>. Finally, <a href="http://bulknews.vox.com/">miyagawa</a>'s <a href="http://yapc10.org/yn2009/talk/2018">Build a desktop application with Perl, HTTP::Engine, SQLite and jQuery</a> talk was exciting. I've been working toward using his <a href="">perl-app-builder</a> for TAEB since he <a href="http://remediecode.org/2009/06/binary-builds-for-os-x-leopard-users.html">blogged about it</a>. The talk showed how we can write standalone, web-based, GUI apps, in Perl, without inflicting CPAN on our users. Because Perl has been a web language ever since the web has had languages, the tools for generating HTML, serving pages, etc. are very good. The traditional GUI tools are mediocre at best. They are why my applications have been strictly web- and curses-based. <h3>Thanks!</h3> Thank you Robert Blackwell, Casey West, Tom Moertel, Dan Wright, and anyone else who helped! This YAPC was fantastic. If you have never been to YAPC, you should seriously consider going to the next one! 2009-06-28 Shawn M Moose at YAPC::NA 2009 http://sartak.org/2009/06/shawn-m-moose-at-yapc-na-2009.html <a href="http://yapc10.org/yn2009/">YAPC::NA 2009</a> begins Monday; it's not too late to sign up! Don't miss the affectionately-titled <a href="http://yapc10.org/yn2009/talks/tag/moose">Moose track</a>. <h3>Extending Moose talk</h3> I'm giving a 50 minute talk about <a href="http://yapc10.org/yn2009/talk/1880">Extending Moose for Applications</a>. The content will be similar to the <a href="http://sartak.org/mmop.html">Moose's MOP</a> series, though focusing more on how all the metaprogramming features work together in harmony. I've been writing the articles so I can learn how to teach the concept and utility of a metaobject protocol. I'm still writing the talk, but you can have a look at what I have so far: <a href="http://sartak.org/talks/yapc-na-2009/extending-moose/extending-moose.pdf">[pdf]</a> <a href="http://sartak.org/talks/yapc-na-2009/extending-moose/extending-moose.key">[keynote source]</a>. Suggestions for improvement or clarification would be very welcome. I have the "script" included as notes at the bottom of each slide, so you can learn even without me there speaking at you. If you intend to attend the talk (or the Moose course on Sunday) you should probably not read it. I stick to the script, terrible jokes and all. When it is done and presented, I'll post about the final version. <h3>Moose class</h3> Due to health problems, Dave Rolsky will sadly <a href="http://blog.urth.org/2009/06/i-wont-be-at-yapc-10.html">not be attending YAPC this year</a>. He was scheduled to give an eight hour class about Moose on Sunday. It would be a lot of hassle for organizers and disappointment for students to cancel the class, so I stepped up to present the bulk of it. <a href="http://jrock.us/">Jon Rockway</a> will be helping as well. I don't make great use of Moose's types, but he does, so we are forcing him to teach the types portion of the class. I will also be presenting my Extending Moose talk at the end of the class (if we turn out to be short on time). This way you can see Matt Trout's <a href="http://yapc10.org/yn2009/talk/1975">Future of DBIx::Class</a> or Christopher Nehren's <a href="http://yapc10.org/yn2009/talk/2016">CLI apps don't have to suck</a>. Both will probably be Moose-heavy and certainly worth seeing. Or.. you can sleep in! Please let me know if you have any questions about the talk or class. I've given a few talks before, but never an extended class, so it will certainly be interesting! I look forward to seeing many of you next week. I'll try to squeeze in a post or eight about talks and all the other good stuff at YAPCs. 2009-06-17 The Design of Parameterized Roles http://sartak.org/2009/05/the-design-of-parameterized-roles.html There has been a lot of good discussion about roles lately, due to <a href="http://www.modernperlbooks.com/cgi-bin/mt/mt-search.cgi?blog_id=1&tag=roles&limit=20">chromatic</a> and <a href="http://use.perl.org/~Ovid/journal">Ovid</a>. We're starting to attract people from outside of not only the <a href="http://moose.perl.org">Moose</a> and <a href="http://dev.perl.org/perl6/">Perl 6</a> communities, but outside of the greater Perl community. I'm thrilled, because roles are an excellent complement to inheritance, and deserve to be widely adopted. In <a href="http://www.modernperlbooks.com/mt/2009/04/the-why-of-perl-roles.html">The Why of Perl Roles</a>, chromatic quickly describes parametric roles, also known as parameterized roles. A parameterized role is a role whose application can be customized. Such a role accepts parameters at composition time to alter its behavior any way it wishes. This increases the flexibility of the role, fostering opportunities for greater code reuse. <h3>Example</h3> My <a href="http://taeb.sartak.org">NetHack bot TAEB</a> has a <a href="https://github.com/sartak/TAEB/blob/master/lib/TAEB/Action/Role/Item.pm"><tt>TAEB::Action::Role::Item</tt></a> parameterized role. It accepts one parameter, a list of names for item attributes. These attributes (which specify a type constraint of <a href="http://p3rl.org/NetHack::Item"><tt>NetHack::Item</tt></a> and a custom "provided" trait) are added to the class upon composition. The role provides a few other things as well: a <tt>current_item</tt> attribute and a method for dealing with missing-item exceptions. The <tt>current_item</tt> attribute's value is a lazy default, pointing to the first item in the parameterized list of attribute names. This is an application of the well-trodden Don't Repeat Yourself principle — each class does not have to repeat its item attribute names. Parameterized roles are pretty good enablers of DRY! Since NetHack is a very item-oriented game, this role has many consumers. The <a href="https://github.com/sartak/TAEB/blob/master/lib/TAEB/Action/Dip.pm"><tt>TAEB::Action::Dip</tt></a> consumer specifies <tt>item</tt> and <tt>into</tt> for the list of attribute names, so it gets those two attributes (as well as whatever else the role provides). Likewise, <a href="https://github.com/sartak/TAEB/blob/master/lib/TAEB/Action/Wield.pm"><tt>TAEB::Action::Wield</tt></a> gets a <tt>weapon</tt> attribute. Most of the consumers specify nothing for this parameter, so they get the default list of <tt>item</tt>. Consuming a parameterized role without specifying any parameters is syntactically identical to applying vanilla roles, which is a (minor) convenience. In older versions of the code, <tt>TAEB::Action::Role::Item</tt> was not parameterized. This role always gave you a single attribute named <tt>item</tt>. The <tt>into</tt> attribute of <tt>TAEB::Action::Dip</tt> was a second-class citizen. There are a number of ways a second-class item attribute can go wrong, such as not having a type constraint, or not being visible to <tt>current_item</tt>'s lazy default. It also prevented us from renaming the <tt>item</tt> attribute to, say, <tt>weapon</tt> for <tt>TAEB::Action::Wield</tt>. We drastically improved the code's clarity and expressiveness by parameterizing the role. <h3>Design</h3> In December, I wrote the <a href="http://p3rl.org/MooseX::Role::Parameterized"><tt>MooseX::Role::Parameterized</tt></a> extension. I had grown tired of people (primarily myself!) putting off coding because the proper implementation of whatever demanded parameterized roles. I am not exaggerating: <ul class="irc"> <li>"hoping once we have parameterised roles we can do that"</li> <li>"you will have [...] parameterized roles which will allow such things very easily"</li> <li>"let's see what the parameterized roles end up giving us"</li> <li>"once we have parameterised roles I shall be providing one to produce all three trivially"</li> <li>"parameterized roles will probably do all that you want"</li> <li>"I'm hoping parameterised roles will let me get rid of that"</li> <li>"what's the status of parameterized roles? I could really use them"</li> <li>"has there been any progress on parameterized roles?"</li> <li>"we don't have parameterized roles yet?!"</li> <li>"are parametrized roles somewhere on the roadmap"</li> <li>"that sounds like parameterized roles, which someone should implement some time"</li> </ul> I think the biggest roadblock in the way of implementing a parameterized role system was Moose's lack of support for anonymous roles. So I did that hard part first then wrote <a href="http://p3rl.org/MooseX::Role::Parameterized">MooseX::Role::Parameterized</a>. Designers of parameterized role systems (perhaps three or four of us by now?) must decide how much freedom each role is given. How much parameterizability is permissible? In <tt>MooseX::Role::Parameterized</tt>, and <a href="http://rakudo.org/2009/01/parametric-roles.html">Perl 6's parametric roles</a>, there are no limitations. You can parameterize anything. Maximizing your user's freedom is certainly laudable, but I am not convinced that it is completely beneficial. There is value in stricture, for example, to improve the efficacy of class/role browsers. I will leave such meditations for the next parameterized role designer. At least for Moose, we have several more chances to get the design right; there are plenty of other namespaces available on CPAN: <tt>MooseX::Role::Parametric</tt>, <tt>MooseX::Role::Parameterizable</tt>, <tt>MooseX::Role::Parameterised</tt>, and so on. <tt>MooseX::Role::Parameterized</tt> works, in short, by executing a procedure for each role application. This procedure takes as an argument a "parameters" object. It is executed in the dynamic extent of an anonymous role metaobject. All of the usual role keywords (<tt>has</tt>, <tt>around</tt>, <tt>requires</tt>, etc) operate on this anonymous role. The procedure may use the "parameters" object in building up the role using these keywords. This is also a reasonable description of how vanilla roles are created, from a backstage point-of-view. The primary difference is that the procedure is only ever executed once for vanilla roles, whereas it is executed potentially many times, building up a different role metaobject each time, for parameterized roles. This difference is precisely what makes the syntax of <tt>MooseX::Role::Parameterized</tt> slightly unusual, at least in the context of core Moose: <pre class="perl code_snippet"> package Counter; use MooseX::Role::Parameterized; parameter name => ( isa => 'Str', required => 1, ); </pre> You explicitly specify which parameters the role can take. This improves introspectability, and makes it immediately obvious (if you've written Moose code) how to specify that a parameter is required, or has a type constraint; default value; or predicate, etc. <pre class="perl code_snippet"> role { my $p = shift; my $name = $p->name; has $name => ( is => 'rw', isa => 'Int', ); method "increment_$name" => sub { my $self = shift; $self->$name($self->$name + 1); }; }; </pre> The <tt>role</tt> keyword exists for specifying the aforementioned role-building procedure. This is the unusual syntax. <a href="http://p3rl.org/MooseX::Declare"><tt>MooseX::Declare</tt></a> could improve upon this a lot, once some tuits are spent. The <tt>method</tt> keyword is required (instead of the usual <tt>sub name { ... }</tt>) because Perl handles named, inner subroutines in a way that is not useful for this design. The details are many, but mostly unimportant. The salient point is that such subroutines do not properly close over <tt>$p</tt>, rendering them unparameterizable. The anonymous subroutine passed to <tt>method</tt> can properly capture <tt>$p</tt>. The <tt>method</tt> keyword has a fringe benefit over <tt>sub</tt> in easing the parameterization of the method's name. <h3>Current and future work</h3> I recently fixed <a href="http://rt.cpan.org/Public/Bug/Display.html?id=45393">a bug</a> in <tt>MooseX::Role::Parameterized</tt> that prevented parameterized roles from participating in role summation. Role summation is what provides protection against method name conflicts at compile time, among other things. Being unable to partake in composition limited parameterized roles' usefulness. I'm very happy to have fixed the one crippling issue with this extension. The fix involved a bait-and-switch hook in Moose's role summation code. The hook is fairly contrived, but it might possibly be useful to other role extensions. I'm currently noodling around with the partial application of parameterized roles. Currently, if a role consumes a parameterized role, it must specify all of the required parameters. In fact, the parameterized role is bound (the procedure is executed) at that moment. <tt>MooseX::Role::Parameterized</tt> does not care about whether it is being consumed by a class or a role. Instead, I'd like the consuming role to be able to specify some of the parameters. Any unbound parameters will be parameters in the composite role. The only blocker stopping me from working on partial application of parameterized roles is a use case. Adding complexity for no reason is clumsy. 2009-05-12 Perl::Critic::Dynamic::Moose http://sartak.org/2009/05/perl-critic-dynamic-moose.html In <a href="/2009/04/the-new-moose-warning-and-the-new-moose-critic.html">my previous post</a>, I wrote "What we need is a set of Perl::Critic policies for Moose." I've put my money where my mouth is and implemented five of them. <h3>CPAN?</h3> <tt>Perl::Critic::Dynamic::Moose</tt> is not on CPAN yet because <tt>Perl::Critic</tt> does not currently distinguish between "safe" (static) and "possibly unsafe" (dynamic) policies. <tt>Perl::Critic</tt> usually works by statically inspecting some Perl document. This allows users to <a href="http://perlcritic.com/">critique untrusted code</a>. Most of the policies I wrote are dynamic. These dynamic policies work by executing the code to be critiqued, then analyzing any new metaclasses that were created. While this limits their utility to analyzing only trusted code, this dynamicism does magnify analytic potential. Instead of inspecting <a href="http://p3rl.org/PPI">PPI</a> DOMs, these policies can inspect any and every metaobject in the system. A large part of why the metaobject protocol exists is for sane and consistent inspection, so these dynamic policies work well. Anyway, <tt>Perl::Critic</tt> uses <a href="http://p3rl.org/Module::Pluggable"><tt>Module::Pluggable</tt></a> to unconditionally load every module in the <tt>Perl::Critic::Policy</tt> namespace. It would be harmful to add unsafe modules to that namespace, so I am going to make sure that a new <tt>Perl::Critic</tt> that runs only safe policies is released before I begin publishing dynamic policies. It should be a very simple matter of programming... <h3>Perl::Critic::DynamicMoosePolicy</h3> <tt>Perl::Critic::DynamicMoosePolicy</tt> is a base class for dynamic Moose policies. It's written, of course, with Moose (extending <tt>Perl::Critic::Policy</tt> using <a href="http://p3rl.org/MooseX::NonMoose"><tt>MooseX::NonMoose</tt></a>). It does the things that every dynamic Moose policy needs to do: <ol> <li>Compile the document being analyzed</li> <li>Find all the metaclasses that were just created</li> <li>Call your policy's <tt>violates_metaclass</tt> for each metaclass</li> </ol> Ideally, policies would just implement <tt>violates_metaclass</tt> and it would all just work. However, there are (seemingly always) some details. Since this analysis is entirely divorced from the source code, it is impossible (in general) to provide a useful file and line number pointing to the violation. Instead of making each policy provide an arbitrary location in the document for violations, <tt>DynamicMoosePolicy</tt> just does it for you. If you're combining PPI and metaobject inspection, then you can of course provide a more useful violation location when you have one. Instead of a location, I recommend that you be as detailed as possible in your error report (include the class name!) <a href="http://p3rl.org/Perl::Critic::DynamicPolicy"><tt>Perl::Critic::DynamicPolicy</tt></a> runs each policy in a forked process, to prevent the analyzed code from mucking with the analyzing process's operation. Dynamic policies communicate violations with perlcritic via a <a href="http://perldoc.perl.org/functions/pipe.html">parent-child pipe</a>, with all violation objects frozen and thawed with <a href="http://p3rl.org/Storable">Storable</a>. This means your violations cannot include code references (such as those created by <a href="http://p3rl.org/Scalar::Defer"><tt>Scalar::Defer</tt></a>), or anything else that <tt>Storable</tt> cannot handle. I spent a while scratching my head trying to figure out what was causing this error: <tt>Cannot restore overloading on HASH(0xc42f0c) (package <unknown>) at blib/lib/Storable.pm (autosplit into blib/lib/auto/Storable/thaw.al) line 415</tt> It turned out to be the <tt>PPI::Document</tt> object that the violation was using as an arbitrary location. Specifying the first element in the document made that error go away. I'm not going to ask too many questions here, just smile and nod. <h4>augment/inner</h4> Policies may define a <tt>applies_to_metaclass</tt> method that filters for only the metaclasses that sensibly apply. I utilized the rarely-seen <a href="http://p3rl.org/Moose::Cookbook::Basics::Recipe6"><tt>augment</tt> and <tt>inner</tt></a> for this method (as well as <tt>default_themes</tt>). The default implementation of <tt>applies_to_metaclass</tt> is: <pre class="perl code_snippet"> sub applies_to_metaclass { 'Class::MOP::Class', inner() } </pre> If the policy wants to analyze only role metaobjects, then they can ignore the method's inverse nature by writing the standard: <pre class="perl code_snippet"> sub applies_to_metaclass { 'Moose::Meta::Role' } </pre> If the policy wants to analyze only class metaobjects, then they do not have to augment this method. The parent's <tt>inner</tt> in such cases is a no-op, returning the empty list. However, if a policy wants to analyze both classes and roles (which I suspect happens when the actual subject for analysis is attributes and methods), the policy can include: <pre class="perl code_snippet"> augment applies_to_metaclass => sub { 'Moose::Meta::Role' }; </pre> <tt>augment</tt> signals that the method resolution order descends the class hierarchy: grandparent -> parent -> child. <tt>inner</tt> calls the next most specific method. The end result of calling the above method is the list <tt>('Class::MOP::Class', 'Moose::Meta::Role')</tt>. <h3>PCP::DynamicMoose::ProhibitPublicBuilder</h3> I wrote this policy first because I knew this area of the metaobject protocol well. Any problems were likely caused by the fork-and-eval side, not the inspection side. Pick your battles. Each attribute can have a <tt>builder</tt> method that is called to provide a default value for that attribute. This facilitates changing this default value by subclassing and role application better than the <tt>default</tt> option does. The inspection is straightforward. For each attribute on the class (or role), ensure that, if it has a builder, its name begins with an underscore. The only reason for this is because I sometimes forget my builders would be public API, when they generally should not be. Handling roles for this policy sucks because attributes in roles are just inert hash references. Since there have been threats to improve roles to make use of real objects, I will elide handling roles here for brevity. <pre class="perl code_snippet"> for my $name ($meta->get_attribute_list) { my $attribute = $meta->get_attribute($name); my $builder = $attribute->builder or next; if ($builder !~ /^_/) { my $desc = "Builder method '$builder' of attribute ..."; push @violations, $self->violation($desc, $PL); } } </pre> <tt>get_attribute_list</tt> returns the names of attributes defined in this local class. Inherited attributes are not considered. I figure you would be running these policies on every class and role in your project anyway. <h3>PCP::DynamicMoose::RequireMakeImmutable</h3> Next up is another easy policy. This is by far the simplest. It performs one simple check on the metaclass (<tt>is_immutable</tt>). Elliot Shank wrote <a href="http://p3rl.org/Perl::Critic::Policy::Moose::RequireMakeImmutable">Perl::Critic::Policy::Moose::RequireMakeImmutable</a>, a static policy that checks for <tt>__PACKAGE__->meta->make_immutable</tt>. There is room enough for both a static and a dynamic policy. Not every class can <tt>make_immutable</tt> in that specific way. Class::MOP, for example calls <tt>make_immutable</tt> on all of its classes at once. In other cases, the programmer may have accidentally forgotten to call <tt>make_immutable</tt> on a dynamically-generated class. As with most static policies, <tt>PCP::Moose::RequireMakeImmutable</tt> admirably handles the common cases, but might not handle strange corner cases well. Dynamic policies can fill such gaps. <h3>PCP::Moose::RequireMethodModifiers</h3> Moose provides a rich set of method modifiers: <ul> <li><tt>before</tt></li> <li><tt>after</tt></li> <li><tt>around</tt></li> <li><tt>override</tt> / <tt>super</tt></li> <li><tt>augment</tt> / <tt>inner</tt></li> </ul> There's a method modifier for just about every way you'd want to interact with homonymous methods. If you define a method that is already present through inheritance, then this policy asks you to explicitly use a method modifier. This way, you might be able to catch accidental method overrides. If you do want to completely replace an ancestor's method, then you can use <tt>override</tt> without calling <tt>super</tt>. That too provides some typo protection, since <tt>override</tt> will confirm that an ancestral method exists. This is the first novel dynamic policy; the previous policies could be implemented statically, but this one would have little chance. Let's walk through its entire <tt>violates_metaclass</tt>. <pre class="perl code_snippet"> sub violates_metaclass { my $self = shift; my $class = shift; my @violations; </pre> This policy does not apply to roles, and does not require Moose-specific features (such as, well, roles), so it can use the default <tt>applies_to_metaclass</tt>. We also need a place to store the list of violations, since many methods could be implicitly overriding inherited methods. <pre class="perl code_snippet"> for my $name ($class->get_method_list) { my $method = $class->get_method($name); </pre> We iterate over the class's local methods. <tt>get_method_list</tt> and <tt>get_method</tt> do not look at inherited methods at all. Methods brought in from locally-consumed roles are also included in this list — this is a feature called flattening. (If in your code you do want to consider inheritance, use <tt>get_all_methods</tt> and <tt>find_method_by_name</tt>) <pre class="perl code_snippet"> # override and augment modifiers are always fine. next if $method->isa('Moose::Meta::Method::Overridden') || $method->isa('Moose::Meta::Method::Augmented'); </pre> This quickly excludes two of the five method modifiers. Such modifiers have their own metaclasses for various reasons, not least of which is our usage here: to allow detection that the method is just a modifier. <pre class="perl code_snippet"> # Since we can implicitly override and wrap in the same class, we # need to be a little more careful here. if ($method->isa('Class::MOP::Method::Wrapped')) { my $orig_method = $method->get_original_method; next if $method->associated_metaclass->name ne $orig_method->associated_metaclass->name; } </pre> Handling the other three method modifiers (<tt>before</tt>, <tt>after</tt>, and <tt>around</tt>) is trickier. You can both define a method and wrap it with these three modifiers in the same class. This is useful for, say, running code <tt>before</tt> an attribute's <tt>clearer</tt>. If the method we're wrapping is from a different metaclass, then we must be wrapping an inherited method, which is fine. If we're wrapping a local method, then we need to peel off a layer. We do not need peel off many layers in a loop, since multiple local modifiers coalesce into one layer. <pre class="perl code_snippet"> # Generated methods next if $method->isa('Class::MOP::Method::Generated'); </pre> This one worries me a little. Generated methods are those created for you by Moose. Examples include accessors, <tt>new</tt> (but only after a <tt>make_immutable</tt>), and delegation methods. Since these methods are generated for you, you have no way to indicate that they are overriding an inherited method. Because you can certainly accidentally override an inherited method with a generated method, this check should be optional. <pre class="perl code_snippet"> # XXX: this freaking sucks next if $name eq 'meta' || $name eq 'BUILD' || $name eq 'DEMOLISH'; </pre> This "freaking sucks" for two reasons. <tt>meta</tt> is installed into your local class by Moose. That's fine (though we will eventually support renaming or excluding it). The metaclass of the <tt>meta</tt> method is not <tt>Class::MOP::Method::Generated</tt> like it should be, it's <tt>Moose::Meta::Method</tt>. This misclassification makes it look like a regular method defined by the user. <tt>meta</tt> would receive special consideration even if it were properly labeled as a generated method, because every class gets its own. The other reason this sucks is the hardcoded <tt>BUILD</tt> and <tt>DEMOLISH</tt>. These are special methods with unusual dispatch semantics. Every <tt>BUILD</tt> and <tt>DEMOLISH</tt> method is called in your hierarchy automatically; there is no choice about redispatching to your inherited method. This is a useful feature, because too often do we forget to do this which can silently mess up initialization or destruction. Anyway, this list of special methods should be inspectable from the metaclass, and given their own <tt>Class::MOP::Method</tt> subclass. <pre class="perl code_snippet"> my $next = $class->find_next_method_by_name($name); # Adding new methods is always fine. next if !$next; </pre> Simple enough, right? It finds the inherited method by a given name, if any. If there is none, then it must be an entirely new method with a unique name, which is perfectly fine. <pre class="perl code_snippet"> push @violations, $self->violation( "The '$name' method of class " . $class->name . " does not use a method modifier to" . " override its superclass implementation.", $EXPL); } return @violations; } </pre> That's all. We've exhausted all the acceptable ways for a method to interact with its super-methods, so what remains must be a violation. <h3>PCP::DynamicMoose::ClassOverridesRole</h3> Ahhh, the violation that launched a thousand comments. I've already <a href="/2009/04/the-new-moose-warning-and-the-new-moose-critic.html">summed up the what and the why</a> of this violation. I wrote this policy today. It required keeping track of more metadata in Moose. There exist a handful of <tt>Moose::Meta::Role::Application</tt> classes, which track, among other things, an application of a role to a class. We were creating these objects but then discarding them once the application was completed. Instead, I improved Moose so we now keep track of them. Progress! Let's walk through this implementation as well. <pre class="perl code_snippet"> sub violates_metaclass { my $self = shift; my $class = shift; my @violations; </pre> Familiar enough. <pre class="perl code_snippet"> for my $application ($class->role_applications) { my $role = $application->role; </pre> Every time you consume a role with <tt>with</tt>, it creates an object of class <tt>Moose::Meta::Role::Application::ToClass</tt> and adds it to your metaclass. This object has, among other things, a <tt>role</tt> attribute. As an aside, it might be interesting to make the <tt>does</tt> method return these role application objects. They're certainly true values, and there is precedent: the true value that <tt>can</tt> returns is a code reference. <pre class="perl code_snippet"> for my $method ($role->get_method_list) { next if $application->is_method_excluded($method); next if $application->is_method_aliased($method); </pre> We need to consider every method in the role. If a method has been excluded or aliased, then that is explicit overriding, so the violation does not apply. To allow for this explicit override is exactly why I needed to add role-application tracking to Moose, and why I could not implement this policy until today. <pre class="perl code_snippet"> my $method_object = $class->get_method($method) or next; </pre> We inspect the method's metaobject from the class. Again, we use <tt>get_method</tt> and not <tt>find_method_by_name</tt> because we do not want to consider inheritance. At this point, the method is definitely there, since it hasn't been excluded. What matters now is whether the role provided it or if the class did. <pre class="perl code_snippet"> if ($method_object->isa('Moose::Meta::Role::Method')) { next if $method_object->body == $role->get_method($method)->body; } </pre> Methods of class <tt>Moose::Meta::Role::Method</tt> were originally pulled in from a role. If the method was not provided by a role, then it was a silent class override. The inside statement makes sure that the method was provided from the correct role, not just any role. If you sequentially apply <tt>Role::Dog</tt> then <tt>Role::Tree</tt>, you would want to know about the implicit override of Dog's <tt>bark</tt> method with Tree's. The inside statement also attempts to handle the method coming in from the same role multiple ways. Suppose you have a <tt>Foo::Basic</tt> role with a method <tt>kerplow</tt> that is consumed by both the <tt>Foo::Advanced</tt> and <tt>Foo::Safe</tt> roles. A class that sequentially consumes <tt>Foo::Advanced</tt> then <tt>Foo::Safe</tt> would get the <tt>kerplow</tt> method twice. But since it's the exact same implementation (i.e. the same code reference), it's not worth warning about. I trust that the programmer will utilize <a href="http://p3rl.org/Test::Perl::Critic"><tt>Test::Perl::Critic</tt></a>, so that the divergence of the two <tt>kerplow</tt> methods would become a violation. <pre class="perl code_snippet"> my $class_name = $class->name; my $role_name = $role->name; my $desc = "Class '$class_name' method '$method'" . " implicitly overrides the same method" . " from role '$role_name'"; push @violations, $self->violation($desc, $EXPL); } } return @violations; } </pre> And that's it. <h3>PCP::Moose::ProhibitMultipleWiths</h3> I did write one static policy, which prohibits multiple calls to <tt>with</tt> in one package. Roles provide powerful and safe means of combination, but these are only activated if you actually combine the roles. If you sequentially apply multiple roles to a class, conflicting method names are silently ignored. This could possibly be fixed in the future, but for now we'll have to live with this behavior. At least we can now warn you about it. This implementation is pretty simple as far as <tt>Perl::Critic</tt> policies go. I wrote this one as a static policy because PPI-based analysis is still very worthwhile. It actually would have been easier to implement dynamically. However, as I explained earlier, you would lose out the "location" of the violation — the second <tt>with</tt>. <h3>Conclusion</h3> I'm quite happy with how dynamic Moose policies are coming along. We now have policies for all the ideas I listed in my previous post. The only one I didn't cover, "Declaring a <tt>new</tt> method instead of using the many object construction hooks", was written quickly after my post by Elliot as a static policy. As far as writing more policies, I think it's largely limited by creativity. I was able to write all the policies I wanted. There were some hairy bits, but mostly it was quite pleasant and a novel use of the metaobject protocol. Ovid has submitted a bug to the Moose RT queue: <a href="http://rt.cpan.org/Public/Bug/Display.html?id=45591">Excluded methods should be added to 'requires'</a>. I'm not sure I agree with his assertion that "since <tt>My::Class->meta->does('My::Role::Foo')</tt>, there's a promise that it will provide the <tt>&foo</tt> method". Since this may not make it into core Moose, I invite Ovid to write a policy covering this case. It would look quite similar to the <tt>ClassOverridesRole</tt> policy. If you have ideas for more policies, or (better yet) would like help implementing them, get in touch with me! I'd be thrilled to help you out. The code for the DynamicMoose policies is in <a href="https://jules.scsys.co.uk/gitweb/gitweb.cgi?p=gitmo/Perl-Critic-Dynamic-Moose.git;a=summary">Moose's git repository</a>. Commit bits to this repo are handed out liberally, just ask in <tt>#moose</tt> on <tt>irc.perl.org</tt>. Providing a patch with your request will win you much favor. These policies have not been battle tested on large codebases, only on the microscopic class hierarchies in its test cases. I expect that <tt>DynamicMoosePolicy</tt> will need more work before it is all ready for wide use. Get your bug reports in before I move on to the next interesting project<a href="http://github.com/sartak/Class-Browser/tree/master">.</a> :) 2009-05-03 The New Moose Warning and the New Moose Critic http://sartak.org/2009/04/the-new-moose-warning-and-the-new-moose-critic.html <h4>Role Design</h4> <blockquote>A role is something that classes do. Usually, a role encapsulates some piece of behavior or state that can be shared between classes. It is important to understand that roles are not classes. You cannot inherit from a role, and a role cannot be instantiated. We sometimes say that roles are consumed, either by classes or other roles. A role can be composed into a class. In practical terms, this means that all of the methods and attributes defined in a role are added directly to (we sometimes say "flattened into") the class that consumes the role. These attributes and methods then appear as if they were defined in the class itself. A subclass of the consuming class will inherit all of these methods and attributes. – Dave Rolsky in <a href="http://p3rl.org/Moose::Manual::Roles">Moose::Manual::Roles</a></blockquote> Roles can be composed to create new roles. This is not like inheritance; this operation is actually commutative summation. This does not establish a hierarchy, instead it creates a composite role with the methods, attributes, requirements, etc. of its component roles. If two or more roles in the composition provide different methods with the same name, a conflict is generated. Such conflicts do not immediately issue fatal errors, but they must be resolved at composition time. One of the ways a role consumer can resolve such a conflict is by providing its own implementation of the conflicted method name. The method could do whatever the programmer sees fit, but it is most sensible for this custom implementation to invoke the conflicting methods from the component roles. For example, consider two roles that provide a <tt>validate</tt> method. A class can consume the summation of these two roles. Since the two component roles each attempted to contribute a <tt>validate</tt> method, <tt>validate</tt> is a conflicted method in the composite role. To resolve this conflict, a consuming class must define its own <tt>validate</tt> method. The class's <tt>validate</tt> method should generally call the <tt>validate</tt> method of both roles. However, the class needs some way to disambiguate the two roles' <tt>validate</tt> methods so that it can call each in turn. For this reason, roles support method aliasing. The consumer can alias each role's <tt>validate</tt> methods. Aliasing only adds a new name for a method; it does not change the name of any method. That means the original <tt>validate</tt> methods are still conflicting. The consuming class can provide a <tt>validate</tt> method to resolve this conflict, whose implementation calls each role's aliased <tt>validate</tt>. <pre class="perl code_snippet"> # this "with" sums the two roles then applies one role to the class with ( 'Role::Foo' => { alias => { validate => 'validate_foo' } }, 'Role::Bar' => { alias => { validate => 'validate_bar' } }, ); sub validate { my $self = shift; $self->validate_foo(@_); $self->validate_bar(@_); } </pre> While this may seem like a lot of effort, role design has very nice theoretical and practical properties. Unlike multiple inheritance and mixins, roles issue an error when there is an unresolved naming conflict. The programmer has good tools for resolving conflicts: method aliasing, method exclusion, and method overriding. In our above example, the consuming class could have excluded the <tt>validate</tt> method from one of the two roles so that there would be no conflict (since only one role is providing a method with that name). Finally, the consuming class could have overridden the conflicted <tt>validate</tt> method. The overriding implementation may not need to call the component roles' <tt>validate</tt> methods, perhaps obviating the need for aliasing. The example above used a combination of aliasing and overriding. All of these features are available even when there are no method conflicts, granting additional flexibility to roles. <h4>The New Moose Warning</h4> <a href="http://use.perl.org/~Ovid/journal/">Ovid</a> has written eleven articles on roles since March 4th (as of this posting). Most of them have <a href="http://use.perl.org/~Ovid/journal/38662">praised roles</a> and explained his happiness with how <a href="http://use.perl.org/~Ovid/journal/38785">comprehensible</a> his large <s>class</s> role space has become. However, in <a href="http://use.perl.org/~Ovid/journal/38809">Not All Features Are Useful (Moose Roles)</a>, Ovid describes his frustration at spending a long time trying to discover an unintentional use of the method overriding feature. Which is silent. Marcus Ramberg was the first to suggest the addition a warning to the Moose core when a class overrides a method from a role it consumes. This way we could push the "frontier" of the beloved collision detection a little further, into boundary between roles and classes. The warning (which was uncannily easy to implement, as if it had existed in a past Alces) would read like: <blockquote>The Quux class has implicitly overridden the method (validate) from role Role::Foo. If this is intentional, please exclude the method from composition to silence this warning (see Moose::Cookbook::Roles::Recipe2)</blockquote> I was one of the very few vocal proponents of this new warning. Suggesting to users that they explicitly list which methods they intended to exclude from composition would benefit maintainability. Other developers would not have to guess whether each method override was intentional or not; intention would be evident from the exclusions, or lack thereof. In a large, evolving system, accidental name collisions are bound to happen. I'd much prefer a compile-time warning over spending even ten minutes debugging why my application decided to start connecting to a database named "Shawn Moore". However, many people were unhappy with this warning, because it complained about perfectly valid semantics. Classes overriding local role methods has been in the design of roles since they were conceived as <a href="http://scg.unibe.ch/research/traits">traits</a> by Smalltalk researchers. Due to limitations in Perl's builtin warning system, the warning was also difficult to disable for a particular scope. <a href="http://www.modernperlbooks.com/">chromatic</a> in particular decried this new warning in his article <a href="http://www.modernperlbooks.com/mt/2009/04/the-value-of-a-warning.html">The Value of a Warning</a>. While I continue to disagree with a few of his specific points, the argument he made in a comment on Ovid's <a href="http://use.perl.org/~Ovid/journal/38862">Well, Now I Know (Roles)</a> article struck a chord. <blockquote>The moment you throw a mandatory default warning on two of the four appropriate and specified uses of roles, you've penalized them. You're subtly encouraging people not to use the most important and most powerful features of roles! You're actively discouraging people from taking advantage of alloorphism using well-established and long-recommended design techniques explicitly made safer and more understandable by roles.</blockquote> He's absolutely correct. Even subtly discouraging users from perfectly good designs is unacceptable. I reverted the warning. <h4>The New Moose Critic</h4> One of the suggested ideas was to leave the warning disabled by default. For a while I thought it was quite ridiculous to turn Perl's own warnings on in your class, but leave Moose's own warnings off. After all, we have <a href="http://p3rl.org/Modern::Perl">grown</a> as a community to love <tt>use warnings</tt>; it would be silly to repeat that default-off mistake. The final section of chromatic's <a href="http://www.modernperlbooks.com/mt/2009/04/the-value-of-a-warning.html">The Value of a Warning</a> held the best compromise. I was dead set against warnings that were off by default, so I didn't give the ideas the thorough consideration they deserved. <blockquote>Compare that to optional <a href="http://p3rl.org/Perl::Critic">Perl::Critic</a> warnings. If your project decides that a method length of more than 24 lines is a code smell, you can determine that yourself. If you decide that a specified, implemented, and tested feature of Perl 5 is troublesome (<tt>tie</tt>), you can disallow it yourself locally.</blockquote> What we need is a set of Perl::Critic policies for Moose. Elliot Shank has already started a <a href="http://p3rl.org/Perl::Critic::Moose">Perl::Critic::Moose</a> for, among other things, ensuring (statically) that you make your classes immutable. Moose stores plenty of data in its metaobjects, rich for plunder. Though nearly all of Perl::Critic performs static analysis, there is precedent for <a href="http://p3rl.org/Perl::Critic::Dynamic">dynamic Perl::Critic policies</a>. We could stand on the shoulders of the Perl::Critic giants so as to reuse and contribute to their <a href="http://p3rl.org/Perl::Critic#CONFIGURATION">configuration system</a>, <a href="http://p3rl.org/Test::Perl::Critic">testing modules</a>, and great community. Some easily-implemented dynamic Moose policies (of varying severity) could highlight: <ul> <li>Classes overriding methods from locally consumed roles</li> <li>Classes overriding methods from their superclasses without using method modifiers (including the <a href="http://p3rl.org/Moose#override">override</a> modifier)</li> <li>Declaring <a href="http://p3rl.org/Moose#builder">builder methods</a> without leading underscores</li> <li>Leaving classes mutable</li> <li>Declaring <tt>new</tt> methods instead of using the many <a href="http://p3rl.org/Moose::Cookbook::Basics::Recipe11">object construction hooks</a></li> </ul> I think this is an excellent compromise and I can't wait to begin implementing Moose policies. 2009-04-24 Breaking Sys::Protect http://sartak.org/2009/03/breaking-sys-protect.html <a href="http://p3rl.org/Sys::Protect">Sys::Protect</a> does not bill itself as unbreakable protection, but it's fun to break it anyway. <a href="http://p3rl.org/PadWalker">PadWalker</a> is used as an example of an XS module that could seriously mess with other code. <a href="http://p3rl.org/Math::BigInt::FastCalc">Math::BigInt::FastCalc</a> is specifically allowed by Sys::Protect to perform unsafe operations because its domain is only big integers, it is used by a lot of other modules, and because its XS code has been verified to be safe. <pre class="perl code_snippet"> #!/usr/bin/env perl no strict; use warnings; use Sys::Protect; use Test::More tests => 1; XSLoader::load(bless {}, 'Break::Sys::Protect'); my $password = 'c53eb8f992b4fdf70a03a4d437820028'; my $authenticate = sub { shift eq $password }; is(${PadWalker::closed_over($authenticate)->{'$password'}}, $password); package Break::Sys::Protect; use overload q{""} => sub { return "Math::BigInt::FastCalc" if caller eq 'Sys::Protect'; $_[0] = "PadWalker"; }; </pre> Don't use Sys::Protect. :) edit: I've been told by Sys::Protect's authors that it is a proof of concept for testing Perl code under environments similar to what Google App Engine supports. Such a platform would presumably remove the dangerous opcodes entirely, so vulnerabilities like this couldn't exist. 2009-03-25 Parametric Roles in Perl 5 http://sartak.org/2009/01/parametric-roles-in-perl-5.html Jonathon Worthington just wrote about <a href="http://use.perl.org/~JonathanWorthington/journal/38308">Perl 6's new support for parametric roles</a>. Excellent! <a href="http://p3rl.org/Moose">Moose</a> supports this feature too, through the <a href="http://p3rl.org/MooseX::Role::Parameterized">MooseX::Role::Parameterized</a> extension I wrote a little over a month ago. It has proved to be a very useful pattern. I'm pleased that Perl 6 has it built in. Out of curiosity, I ported the examples Jonathon provided to MXRP. I'll have to work more with rafl to make <a href="http://p3rl.org/MooseX::Declare">MooseX::Declare</a> support something resembling Perl 6's much nicer syntax. We do plan on having syntax for positional parameters in much the same way Perl 6's parametric roles do. <pre class="perl code_snippet"> package Greet; use MooseX::Role::Parameterized; parameter greeting => ( is => 'ro', isa => 'Str', required => 1, ); role { my $p = shift; my $greeting = $p->greeting; method greet => sub { print "$greeting!\n"; }; }; package EnglishMan; use Moose; with Greet => { greeting => "Hello" }; package Slovak; use Moose; with Greet => { greeting => "Ahoj" }; package Lolcat; use Moose; with Greet => { greeting => "OH HAI" }; EnglishMan->new->greet; # Hello! Slovak->new->greet; # Ahoj! Lolcat->new->greet; # OH HAI! </pre> I'll skip the second example because it's contained by the third example. Moose doesn't give you multiple dispatch. sigh! Instead we model the problem with a default value for the transform, which is a code reference. In this way we meet the original requirement of EnglishMan and Lolcat not needing to provide a nominative->accusative transform. <pre class="perl code_snippet"> package Request; use MooseX::Role::Parameterized; parameter statement => ( is => 'ro', isa => 'Str', required => 1, ); parameter transform => ( is => 'ro', isa => 'CodeRef', default => sub { sub { $_[0] }, # identity function }, ); role { my $p = shift; my $statement = $p->statement; my $transform = $p->transform; method request => sub { my ($self, $object) = @_; print "$statement " . $transform->($object) . "?\n"; }; }; package Language::Slovak; sub accusative { my $nom = shift; (my $acc = $nom) =~ s/a$/u/; return $acc; } package EnglishMan; use Moose; with Request => { statement => "Please can I have a" }; package Slovak; use Moose; with Request => { statement => "Prosim si", transform => \&Language::Slovak::accusative, }; package Lolcat; use Moose; with Request => { statement => "I CAN HAZ" }; EnglishMan->new->request("yorkshire pudding"); Slovak->new->request("boravicka"); Lolcat->new->request("CHEEZEBURGER"); </pre> I have no conclusion, except that Perl 5 hasn't been stagnant, though its new features can be a lot more verbose than in Perl 6. 2009-01-17 Devel::REPL: Now with Multiline Support http://sartak.org/2007/09/devel-repl-now-with-multiline-support.html I've been using <a href="http://p3rl.org/Devel::REPL">Devel::REPL</a> for a while now. Like all good modules (<a href="http://p3rl.org/Perl::Critic">Perl::Critic</a>, <a href="http://p3rl.org/POE">POE</a>, <a href="http://p3rl.org/Plagger">Plagger</a>, etc), it's very extensible. <a href="http://chainsawblues.vox.com/library/post/a-perl-read-excute-print-loop-repl.html">Devel::REPL's design</a> is worth studying: keep a simple core and ship all the fancy behavior as plugins. <a href="http://p3rl.org/Moose">Moose</a> amplifies the power and convenience of this design with roles, method modifiers, and general awesomeness. There are plugins to dump output with <a href="http://p3rl.org/Data::Dump::Streamer">Data::Dump::Streamer</a>, enable tab completion of the current lexical environment and loaded modules, save input history across sessions, and more. However, if you dabble in other P-languages such as Python and Ruby (know thy enemy.. honest!) you'll find yourself wanting more out of Devel::REPL. Let's take the example of writing a factorial function in python: <pre class="perl code_snippet"> % python Python 2.3.5 (#1, Dec 7 2006, 14:50:51) [GCC 4.0.1 (Apple Computer, Inc. build 5363) (+4864187)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> def fact(n): ... if n < 2: ... return 1 ... return n * fact(n - 1) ... >>> fact(10) 3628800 </pre> in irb: <pre class="code_snippet"> % irb irb(main):001:0> def fact(n) irb(main):002:1> if n < 2 irb(main):003:2> 1 irb(main):004:2> else irb(main):005:2* n * fact(n-1) irb(main):006:2> end irb(main):007:1> end => nil irb(main):008:0> fact 10 => 3628800 </pre> in lua: <pre class="code_snippet"> % lua Lua 5.1.2 Copyright (C) 1994-2007 Lua.org, PUC-Rio > function fact(n) >> if n < 2 then >> return 1 >> else >> return n * fact(n-1) >> end >> end > =fact(10) 3628800 </pre> and in Devel::REPL: <pre class="code_snippet"> % re.pl $ sub fact { Compile error: Missing right curly or square bracket at (eval 60) line 8, at end of line syntax error at (eval 60) line 8, at EOF </pre> D'oh. OK, let's try again.. <pre class="code_snippet"> % re.pl $ sub fact { my $n = shift; return 1 if $n < 2; $n * fact($n - 1) } $ fact 10 3628800 </pre> Well, that works in this case, but one big line of code quickly becomes unmanageable. Recently I had the idea to use <a href="http://p3rl.org/PPI">PPI</a> to figure out if the current line of code is complete. PPI::Dumper quickly confirmed that I can detect the most important case: a PPI::Structure that doesn't have both a ->start and ->finish. Structures encompass { {nested { blocks } } }, (parentheses), [array indexing], {hash indexing}, and so on. Hopefully future versions of PPI will be able to figure out that, say, an s/// or quoted string is incomplete.Here's what the factorial example looks like with the MultiLine::PPI plugin. <pre class="code_snippet"> % re.pl $ load_plugin 'MultiLine::PPI' 1 $ sub fact { > my $n = shift; > return 1 if $n < 2; > $n * fact($n - 1); > } $ fact 10 3628800 </pre> I believe Devel::REPL is the only Perl REPL that can do this. Hooray! :) You can change the default prompts with my FancyPrompt plugin. Here's how I actually have my prompt (code is slightly different to show off nesting). It may look like "ugh", but it's actually really nice when you're in the driver's seat. <pre class="code_snippet"> % re.pl > sub fact { >> my $n = shift; >> if ($n < 2) { >>> return 1; >>> } >> else { >>> return $n * fact($n-1); >>> } >> } > fact 10 3628800 </pre> MultiLine::PPI hasn't been CPANed yet, but you can get it (and other new plugins) from <a href="http://git.shadowcat.co.uk/gitweb/gitweb.cgi?p=p5sagit/Devel-REPL.git;a=summary">the Devel::REPL git repository</a>. I'll continue stealing good features from other REPLs. :) 2007-09-21