tag:blogger.com,1999:blog-4376947664270917518.post2056072223687490771..comments2023-05-21T13:47:49.178+01:00Comments on Adventures in Programming Languages and Semantics: If concurrency is easy, you're either absurdly smart or you're kidding yourselfGianhttp://www.blogger.com/profile/04384217245734242968noreply@blogger.comBlogger15125tag:blogger.com,1999:blog-4376947664270917518.post-34682471126285592192020-10-29T12:00:30.226+00:002020-10-29T12:00:30.226+00:00Thanks for the blog loaded with so many informatio...Thanks for the blog loaded with so many information. Stopping by your blog helped me to get what I was looking for. <a href="https://fromatour.uk/index.php/blogs/3900/18385/problems-you-need-to-avoid-to-progress-in-your-english-classes" rel="nofollow">어린이 영어</a><br />PEPECUSTOMhttps://www.blogger.com/profile/07243897491429605557noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-38249090315451921362009-08-13T17:39:25.958+01:002009-08-13T17:39:25.958+01:00I agree with the spirit of your post which I inter...I agree with the spirit of your post which I interpret as "be humble when writing concurrent code". It is very easy to get over-confident about one's understanding of concurrent code, only to get slammed later by a use case or hardware platform that shows up your bug. This is true, of course, of any program but the risk of these bugs escaping into the wild are much, much greater with concurrent vs. sequential code. <br /><br />What I don't agree with is that there is anything fundamental either in human cognition or even in standard software engineering education that makes concurrency hard. IMHO, the fundamental problem, as I think clord mentioned, is the set of abstractions (threads, locks, etc.) that most programmers have at their disposal. If we forced most sequential programmers to write assembly code instead of depending on high level languages and compilers, we'd have a similar set of problems with software quality & productivity (hey, that already happened!). <br /><br />The clue to the solution is implied in your description of the physical world. Humans can understand concurrency very easily if a few rules are put in place. Physics is a good example of this kind of rule set. Without a good understanding of those rules, we would't be able to function at all in the real world. However, by understanding the basics like gravity and momentum (and a few rules of thumb on predicting the behavior of our fellow humans), we can deal with massive levels of concurrency every day. Think about driving down a multi-lane highway at 60+mph. The slightest error can create a catastrophic collision and yet, with very few exceptions, we all deal with it easily. Some of us can even talk on the phone, text, put on makeup or shave while doing it (yes, I've seen all of those!)<br /><br />So back to concurrency in programming. I think the critical thing we need to do is not to try to solve the general problem of concurrency management but to gravitate toward a standard family of (in your words) "nice simple patterns of interaction to which we can apply some simple mental analogy". Indeed, this is really what we've done in sequential programming decades with abstract notions like a uniform memory area or complex expression evaluation, neither of which exist in real hardware. We've even adopted simple concurrency patterns and probably not even realized it. The UNIX shell is a great example. Every day, people write scripts and commands which use concurrency and synchronization, never running into problems with deadlocks or data races. <br /><br />So the question we should be asking is what are those patterns?Bob Blaineyhttps://www.blogger.com/profile/05367868406434608667noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-2773864300040964752009-08-12T21:21:20.412+01:002009-08-12T21:21:20.412+01:00That wasn't really a very process algebraic fo...That wasn't really a very process algebraic formulation on my part, it was just meant to capture some notion of computation.<br /><br />And yes, that was basically meant to illustrate the "connection points" between a computation and its environment. Obviously in the purely functional world (or indeed, any other world where arbitrary side effects can't occur), S = S'.Gianhttps://www.blogger.com/profile/04384217245734242968noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-3538537129804991542009-08-12T21:12:16.543+01:002009-08-12T21:12:16.543+01:00Gian,
For me, when I saw the change in state S -&...Gian,<br /><br />For me, when I saw the change in state S -> S' in your comment:<br /><br />(S,x) ---(some computation)---> (S',y)<br /><br />This got me thinking about side effects. Maybe I do not understand process algebra well enough, but based on my knowledge of programming, this is what I took it to mean.<br /><br />I was glad to see the discussion of functional languages like Haskell come up. I would like to add declarative languages for those who might not already be familiar.<br /><br />ToddTodd Bezenekhttps://www.blogger.com/profile/09818439983653621929noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-31958433808453050322009-08-12T16:50:48.213+01:002009-08-12T16:50:48.213+01:00Thanks Nick. I think you've hit the nail on t...Thanks Nick. I think you've hit the nail on the head with pretty much all of those points. <br /><br />I think it would be fascinating to get your hands on a big corpus of very concurrent software and do some actual statistical analysis of the sorts of tactics that people use to manage the complexity. Sadly, like many things which require a large corpus of software and analysis, I don't have the time!Gianhttps://www.blogger.com/profile/04384217245734242968noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-11120184690156753122009-08-12T16:23:04.263+01:002009-08-12T16:23:04.263+01:00Very well written; you have put to words what many...Very well written; you have put to words what many of us take for granted.<br /><br />I study concurrency, and though I admit it can be very difficult to reason about concurrent systems (in any language), I have always been able to make it work. This feat has been through clean, careful design. Here are the guidelines I use:<br /><br />(1) (as you noted), I try to limit the scope of all computations. This corresponds exactly with good object oriented design.<br /><br />(2) I try to conceptualize the program as dataflow, instead of as control flow. I think this is useful because, if you want to take two parts A and B from your program and put them in two threads, you are immediately thinking of the values from A which contribute to B, and thus are thinking about thread synchronization. It also means that you are thinking about (and presumably trying to avoid) nasty things, such as cyclic communications between threads, which can be a nasty performance killer.<br /><br />(2b) Said another way, conceptualizing your program as data flow is the same as conceptualizing it as a program written in a functional style.<br /><br />(3) Armed with [1] and [2], I then try to frame the overall architecture as many instantiations of simple patterns: producer/consumer, software pipelines, worklists, etc. If you're thinking about muteces and semaphores, you're thinking at too low a level; think instead about concurrent queues. <br /><br />(4) Finally, I have a bag of tricks to simplify otherwise complicated multithreaded systems. Do two threads need to share a data structure? Maybe you can privatize it (or otherwise ensure that two threads are operating on disjoin halves of it), thus eliminating many inter-thread synchronizations. Does a value needs to be transferred from one thread to the next? Sometimes its faster and easier to recompute it. etc. etc. etc.<br /><br />When I do all this, things seem to work out well. Let me say explicitly, though, that this refers to systems that were developed from day 0 with concurrency in mind.<br /><br />Another thing to note is that all the patterns I mentioned are ususally considered high-level patterns for coarse grained parallelism. I very much disagree with this, and I emphasize that they can be applied to exploit fine-grained parallelism too.<br /><br />But yeah, I'm going to subscribe to your blog.Nick Johnsonhttps://www.blogger.com/profile/09742551843387605596noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-32761472343172226332009-08-12T02:55:34.480+01:002009-08-12T02:55:34.480+01:00clord, That sounds like it deserves to be nailed t...clord, That sounds like it deserves to be nailed to the door of something.Gianhttps://www.blogger.com/profile/04384217245734242968noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-69524997790288584612009-08-12T01:03:33.419+01:002009-08-12T01:03:33.419+01:00I disagree completely. Humans start out learning c...I disagree completely. Humans start out learning concurrency, and then we hammer on them for decades to make them rigourous and serial in their thinking. It is then no surprise that they fall on their face when we ask them to solve the same problems, but in parallel algorithms; to think non-sequential.<br /><br />I believe computer languages are at fault for failing to describe the sort of richness that the human mind is capable of. As someone involved with a major corporation working on compilers for big metal like roadrunner, I'd like to hope that I have some say in how the language business will fix this. <br /><br />But don't rely on the companies, or the few of us in them who want to tackle this at the language level. Embrace languages that do parallelism right. If a language constrains side-effects, embrace it, because that's what gives us the most problems. the side-effect free languages like Haskell will rule the world as the number of cores increases.Christopher Lordhttps://www.blogger.com/profile/01359289674135226502noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-75839966576200140952009-08-11T22:17:01.466+01:002009-08-11T22:17:01.466+01:00Also, I'm not sure how side-effects came up th...Also, I'm not sure how side-effects came up there. I used "side effect" to mean "an action being communicated on a channel" in the process algebraic sense, not in the my-programming-language-doesn't-have-side-effects kind of way.Gianhttps://www.blogger.com/profile/04384217245734242968noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-67467042460032031232009-08-11T22:13:20.539+01:002009-08-11T22:13:20.539+01:00Andy, that's a class of problems that decompos...Andy, that's a class of problems that decompose well into simple models of concurrency.<br /><br />You've basically taken what I've said in the abstract and you're making it very, very concrete before responding. Concurrency is difficult with just a pencil and a piece of paper! Don't worry about the tools or languages or technologies. Write down some terms in a process calculus. Without using any mechanical methods (i.e. without calculating the result), try to conceive of the entirety of the behaviour in that system. It's hard! There's no mutable state there - just some actions.<br /><br />So, the point I made in the comments was that methods that permit us to have a good mental model of the behaviour of a system are great, but just storing concurrent behaviours in your head in such a way that you can interrogate their behaviour with reasonable accuracy is hard! When we have metaphors which allow us to hide some of the complexity, it gets easier.<br /><br />Or maybe it's all easy for you, in which case, you're probably absurdly smart :)Gianhttps://www.blogger.com/profile/04384217245734242968noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-84717730897724199232009-08-11T22:07:22.220+01:002009-08-11T22:07:22.220+01:00VS, yes, that's exactly what I'm saying. ...VS, yes, that's exactly what I'm saying. I do wonder if learning concurrency first might make life easier, however, I present an interesting counter-example:<br /><br />A supervisor of mine came to programming via process algebras, that is, he learned concurrency first. He has been publishing papers on concurrency theory for the better part of 20 years, and has probably been programming for around half that time. He still remarks on the incredible difficulty of maintaining a consistent mental model of a system. It was through my conversations with him on this topic that inspired me to write this post.<br /><br />So we're certainly versatile enough creature to teach ourselves to do difficult things. I guess my point is just that the way the universe is structured isn't doing us any favours :)Gianhttps://www.blogger.com/profile/04384217245734242968noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-12095946342111179192009-08-11T20:12:31.577+01:002009-08-11T20:12:31.577+01:00You`re suggesting that the difficulty people have ...You`re suggesting that the difficulty people have with concurrency programming is an innate limitation of the brain and our understanding of causality. Personally, I feel that it`s more likely to be a consequence of the way people learn to program. Put another way, if we taught people how to program concurrency from the start, it might be possible to overcome this perceived limitation.VShttps://www.blogger.com/profile/15578765210945400666noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-47805494463612909922009-08-11T17:55:33.102+01:002009-08-11T17:55:33.102+01:00Didn't you just contradict yourself? The prob...Didn't you just contradict yourself? The problem with concurrency is side-effects. No side effects, no problem. If there's no mutable state then there's no race conditions, no need for locks on a 'critical section'. It's the same way as GPUs work. Have you ever written HLSL? That's massive concurrency at its best. You should really look into VVVV (.org) They figured out using HLSL concurrency for graphics, as well as networked concurrency with boygrouping.<br /><br />Sure, concurrency is hard in C++ or Java, you just need a better tool.Unknownhttps://www.blogger.com/profile/15081652103882556809noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-14979898943349333362009-08-11T17:06:47.012+01:002009-08-11T17:06:47.012+01:00Oh, there are certainly some brilliant approaches ...Oh, there are certainly some brilliant approaches that yield good results. I'll quote myself from above though:<br /><br />"Now, before you jump up and down and tell me that "insert-your-favourite-concurrency-model-here addresses these problems", I know there are strategies for managing the complexity of these tasks. I did concurrency theory for a living, so I know some approaches are better than others. My point here was essentially to illustrate why it is important to manage complexity when doing concurrency. It's definitely not just a case of "you can't do concurrency so that means you are stupid". I'm arguing that pretty much everyone is bad at concurrency beyond some simple patterns of interaction that we cling to because we can maintain a good mental model by using them."Gianhttps://www.blogger.com/profile/04384217245734242968noreply@blogger.comtag:blogger.com,1999:blog-4376947664270917518.post-39998432315580797752009-08-11T17:02:56.688+01:002009-08-11T17:02:56.688+01:00You didn't mention Clojure. Or Erlang. Or Hask...You didn't mention Clojure. Or Erlang. Or Haskell. Or Mozart Oz. etc. Curious how these don't elegantly address what you're talking about?Anonymousnoreply@blogger.com