Kean专题(4)—Concurrent_Programming
本帖最后由 作者 于 2009-5-17 12:35:37 编辑一、在AutoCAD使用异步工作流简化并发程序设计(F#)
January 25, 2008
Using F# Asynchronous Workflows to simplify concurrent programming in AutoCAD
In the last post we saw some code that downloaded data - serially - from a number of websites via RSS and created AutoCAD entities linking to the various posts.
As promised, in today's post we take that code and enable it to query the same data in parallel by using Asynchronous Workflows in F#. Asynchronous Workflows are an easy-to-use yet powerful mechanism for enabling concurrent programming in F#.
Firstly, a little background as to why this type of technique is important. As many - if not all - of you are aware, the days of raw processor speed doubling every couple of years are over. The technical innovations that enabledMoore's Law to hold true for half a century are - at least in the area of silicon-based microprocessor design - hitting a wall (it's apparently called the Laws of Physics :-). Barring some disruptive technological development, the future gains in computing performance are to be found in the use of parallel processing, whether via multiple cores, processors or distributed clouds of computing resources.
Additionally, with an increasing focus on distributed computing and information resources, managing tasks asynchronously becomes more important, as information requests across a network inevitably introduce a latency that can be mitigated by the tasks being run in parallel.
The big problem is that concurrent programming is - for the most-part - extremely difficult to do, and even harder to retro-fit into existing applications. Traditional lock-based parallelism (where locks are used to control access to shared computing resources) is both unwieldy and prone to blocking. New technologies, such as Asynchronous Workflows and Software Transactional Memory, provide considerable hope (and this is a topic I have on my list to cover at some future point).
Today's post looks at a relatively simple scenario, in the sense that we want to perform a set of discrete tasks in parallel, harnessing those fancy multi-core systems for those of you lucky enough to have them (I'm hoping to get one when I next replace my notebook, sometime in March), but that these tasks are indeed independent: we want to wait until they are all complete, but we do not have the additional burden of them communicating amongst themselves or using shared resources (e.g. accessing shared memory) during their execution.
We are also going to be very careful only to run parallel tasks unrelated to AutoCAD. Any access made into AutoCAD's database, for instance, needs to be performed in series: AutoCAD is not thread-safe when it comes to the vast majority of its programmatically-accessible functionality. So we're going to run a set of asynchronous, parallel tasks to query our various RSS feeds, and combine the results before creating the corresponding geometry in AutoCAD. This all sounds very complex, but the good (actually great) news is that Asynchronous Workflows does all the heavy lifting. Phew.
Here's the modified F# code// Use lightweight F# syntax
#light
// Declare a specific namespace and module name
module MyNamespace.MyApplication
// Import managed assemblies
#I @"C:\Program Files\Autodesk\AutoCAD 2008"
#r "acdbmgd.dll"
#r "acmgd.dll"
open Autodesk.AutoCAD.Runtime
open Autodesk.AutoCAD.ApplicationServices
open Autodesk.AutoCAD.DatabaseServices
open Autodesk.AutoCAD.Geometry
open System.Xml
open System.Collections
open System.Collections.Generic
open System.IO
open System.Net
open Microsoft.FSharp.Control.CommonExtensions
// The RSS feeds we wish to get. The first two values are
// only used if our code is not able to parse the feed's XML
let feeds =
[ ("Through the Interface",
"http://blogs.autodesk.com/through-the-interface",
"http://through-the-interface.typepad.com/through_the_interface/rss.xml");
("Don Syme's F# blog",
"http://blogs.msdn.com/dsyme/",
"http://blogs.msdn.com/dsyme/rss.xml");
("Shaan Hurley's Between the Lines",
"http://autodesk.blogs.com/between_the_lines",
"http://autodesk.blogs.com/between_the_lines/rss.xml");
("Scott Sheppard's It's Alive in the Lab",
"http://blogs.autodesk.com/labs",
"http://labs.blogs.com/its_alive_in_the_lab/rss.xml");
("Lynn Allen's Blog",
"http://blogs.autodesk.com/lynn",
"http://lynn.blogs.com/lynn_allens_blog/index.rdf");
("Heidi Hewett's AutoCAD Insider",
"http://blogs.autodesk.com/autocadinsider",
"http://heidihewett.blogs.com/my_weblog/index.rdf") ]
// Fetch the contents of a web page, asynchronously
let httpAsync(url:string) =
async { let req = WebRequest.Create(url)
use! resp = req.GetResponseAsync()
use stream = resp.GetResponseStream()
use reader = new StreamReader(stream)
return reader.ReadToEnd() }
// Load an RSS feed's contents into an XML document object
// and use it to extract the titles and their links
// Hopefully these always match (this could be coded more
// defensively)
let titlesAndLinks (name, url, xml) =
let xdoc = new XmlDocument()
xdoc.LoadXml(xml)
let titles =
[ for n in xdoc.SelectNodes("//*")
-> n.InnerText ]
let links =
[ for n in xdoc.SelectNodes("//*") ->
let inn = n.InnerText
ifinn.Length > 0 then
inn
else
let href = n.Attributes.GetNamedItem("href").Value
let rel = n.Attributes.GetNamedItem("rel").Value
if href.Contains("feedburner") then
""
else
href ]
let descs =
[ for n in xdoc.SelectNodes
("//*")
-> n.InnerText ]
// A local function to filter out duplicate entries in
// a list, maintaining their current order.
// Another way would be to use:
// Set.of_list lst |> Set.to_list
// but that results in a sorted (probably reordered) list.
let rec nub lst =
match lst with
| a::[] ->
| a::b ->
if a = List.hd b then
nub b
else
a::nub b
| [] -> []
// Filter the links to get (hopefully) the same number
// and order as the titles and descriptions
let real = List.filter (fun (x:string) -> x.Length > 0)
let lnks = real links |> nub
// Return a link to the overall blog, if we don't have
// the same numbers of titles, links and descriptions
let lnum = List.length lnks
let tnum = List.length titles
let dnum = List.length descs
if tnum = 0 || lnum = 0 || lnum <> tnum || dnum <> tnum then
[(name,url,url)]
else
List.zip3 titles lnks descs
// For a particular (name,url) pair,
// create an AutoCAD HyperLink object
let hyperlink (name,url,desc) =
let hl = new HyperLink()
hl.Name <- url
hl.Description <- desc
(name, hl)
// Use asynchronous workflows in F# to download
// an RSS feed and return AutoCAD HyperLinks
// corresponding to its posts
let hyperlinksAsync (name, url, feed) =
async { let! xml = httpAsync feed
let tl = titlesAndLinks (name, url, xml)
return List.map hyperlink tl }
// Now we declare our command
[<CommandMethod("rss")>]
let createHyperlinksFromRss() =
// Let's get the usual helpful AutoCAD objects
let doc =
Application.DocumentManager.MdiActiveDocument
let db = doc.Database
// "use" has the same effect as "using" in C#
use tr =
db.TransactionManager.StartTransaction()
// Get appropriately-typed BlockTable and BTRs
let bt =
tr.GetObject
(db.BlockTableId,OpenMode.ForRead)
:?> BlockTable
let ms =
tr.GetObject
(bt.,
OpenMode.ForWrite)
:?> BlockTableRecord
// Add text objects linking to the provided list of
// HyperLinks, starting at the specified location
// Note the valid use of tr and ms, as they are in scope
let addTextObjects pt lst =
// Use a for loop, as we care about the index to
// position the various text items
let len = List.length lst
for index = 0 to len - 1 do
let txt = new DBText()
let (name:string,hl:HyperLink) = List.nth lst index
txt.TextString <- name
let offset =
if index = 0 then
0.0
else
1.0
// This is where you can adjust:
//the initial outdent (x value)
//and the line spacing (y value)
let vec =
new Vector3d
(1.0 * offset,
-0.5 * (Int32.to_float index),
0.0)
let pt2 = pt + vec
txt.Position <- pt2
ms.AppendEntity(txt) |> ignore
tr.AddNewlyCreatedDBObject(txt,true)
txt.Hyperlinks.Add(hl) |> ignore
// Here's where we do the real work, by firing
// off - and coordinating - asynchronous tasks
// to create HyperLink objects for all our posts
let links =
Async.Run
(Async.Parallel
[ for (name,url,feed) in feeds ->
hyperlinksAsync (name,url,feed) ])
// Add the resulting objects to the model-space
let len = Array.length links
for index = 0 to len - 1 do
// This is where you can adjust:
//the column spacing (x value)
//the vertical offset from origin (y axis)
let pt =
new Point3d
(15.0 * (Int32.to_float index),
30.0,
0.0)
addTextObjects pt (Array.get links index)
tr.Commit()
A few comments on the changes:
Lines 57-62 define our new httpAsync() function, which uses GetResponseAsync() - a function exposed in F# 1.9.3.7 - to download the contents of a web-page asynchronously .
Lines 141-144 define another asynchronous function, hyperlinksAsync(), which calls httpAsync() and then - as before - extracts the feed information and creates a corresponding list of HyperLinks. This is significant: creation of AutoCAD HyperLink objects will be done on parallel; it is the addition of these objects to the drawing database that needs to be performed serially.
Lines 214-217 replace our very simple "map" with something slightly more complex: this code runs a list of tasks in parallel and waits for them all to complete before continuing. What is especially cool about this implementation is the fact that exceptions in individual tasks result in the overall task failing (a good thing, believe it or not :-), and the remaining tasks being terminated gracefully.
Lines 221 and 233 change our code to handle an array, rather than a list (while "map" previously returned a list, Async.Run returns an array).
When run, the code creates exactly the same thing as last time (although there are a few more posts in some of the blogs ;-)
A quick word on timing: I used "F# Interactive" to do a little benchmarking on my system, and even though it's single-core, single-processor, there was a considerable difference between the two implementations. I'll talk more about F# Interactive at some point, but think of it to F# in Visual Studio as the command-line is to LISP in AutoCAD: you can very easily test out fragments of F#, either by entering them directly into the F# Interactive window or highlighting them in Visual Studio's text editor and hitting Alt-Enter.
To enable function timing I entered "#time;;" (without the quotations marks) in the F# Interactive window. I then selected and loaded the supporting functions needed for each test - not including the code that adds the DBText objects with their HyperLinks to the database, as we're only in Visual Studio, not inside AutoCAD - and executed the "let links = ..." assignment in our two implementations of the createHyperlinksFromRss() function (i.e. the RSS command). These functions do create lists of AutoCAD HyperLinks, but that's OK: this is something works even outside AutoCAD, although we wouldn't be able to do anything much with them. Also, the fact we're not including the addition of the entities to the AutoCAD database is not relevant: by then we should have identical data in both versions, which would be added in exactly the same way.
Here are the results:I executed the code for serial querying and parallel querying twice (to make sure there were no effects from page caching on the measurement):
val links : (string * HyperLink) list list
Real: 00:00:14.636, CPU: 00:00:00.15, GC gen0: 5, gen1: 1, gen2: 0
val links : (string * HyperLink) list array
Real: 00:00:06.245, CPU: 00:00:00.31, GC gen0: 3, gen1: 0, gen2: 0
val links : (string * HyperLink) list list
Real: 00:00:15.45, CPU: 00:00:00.46, GC gen0: 5, gen1: 1, gen2: 0
val links : (string * HyperLink) list array
Real: 00:00:03.832, CPU: 00:00:00.62, GC gen0: 2, gen1: 1, gen2: 0
So the serial execution took 14.5 to 15.5 seconds, while the parallel execution took 3.8 to 6.3 seconds.
<p>二、用多义线路径模拟Hatch</p><p>February 21, 2008<br/>Parallelizing robotic AutoCAD hatching with F# and .NET<br/>In the last post we saw some code combining F# with C# to create a random "hatch" - a polyline path that bounces around within a boundary.</p><p>This post extends the code to make use of Asynchronous Workflows in F# to parallelize the testing of points along a segment. In the initial design of the application I decided to test 10 points along each segment, to see whether it remained entirely within our boundary: the idea being that this granularity makes it very likely the segment will fail the test, should it happen to leave the boundary at any point. Not 100% guaranteed, but a high probability event. What this code does is take the 10 tests and queue them up for concurrent processing (where the system is capable of it).</p><p>Asynchronous Workflows - as suggested by the name - were intended to fire off and manage asynchronous tasks (ones that access network resources, for instance). The segment testing activity is actually very local and processor-bound, so it's not really what the mechanism was intended for, but I thought it would be interesting to try. One interesting point: while testing this code I noticed that it actually ran slower on a single processor machine, which is actually quite logical: only one core is available for processing, so the amount of sequential processing is not reduced but the overhead of synchronizing the various tasks is added. So it was fairly inevitable it would take longer. In the post I first talked about Asynchronous Workflows I showed a sample that queried multiple RSS sites for data: even on a single processor machine this was significantly quicker, as parallelizing the network latency led to a clear gain.</p><p>Anyway, as it was slower on this machine I decided only to enable the parallel version of the code in cases where the computer's NUMBER_OF_PROCESSORS environment variable is greater than 1. I checked quickly on a colleague's dual-core machine, and sure enough this variable is set to 2 on his system. I haven't, however, tested the code on a dual- or multi-core system, but I'll be getting a new system in a matter of weeks, which will give me the chance to test it out.</p><p>Here's the the complete project which defines both FB and FBA commands for simple side-by-side comparison.<br/></p> <p>三、编程语言分类</p><p>March 17, 2008<br/>A simple taxonomy of programming languages<br/>Someone asked me recently how I categorize different programming paradigms. I thought it was a very interesting question, so here's what I responded. Please bear in mind that this is very much the way I see things, and is neither an exhaustive nor a formally-ratified taxonomy. </p><p>One way to look at languages is whether they're declarative or imperative: </p><p> Declarative programming languages map the way things are by building up “truths”: this category includes functional programming languages (such as Miranda, Haskell and Erlang) which tend to be mathematical in nature (you define equations) and start with lambda calculus as a foundation. The other main set of declarative languages are logic programming languages (such as Prolog), which start with propositional calculus as a foundation (you declare axioms that build up to a system against which you can run queries). Declarative languages tend to focus on describing the problem to solve, rather than how to solve it. </p><p> Imperative programming languages, on the other hand, are lists of instructions of what to do: I tend to consider procedural programming languages (such as C, COBOL, Fortran and Pascal) as a sub-category which focus on the definition and execution of sub-routines, while some people treat the terms imperative and procedural as synonyms.</p><p>Considering these definitions, object-oriented programming languages (such as Smalltalk and Eiffel) should probably be considered declarative, as conceptually they map real-world objects, but the truth is that the most popular OO languages (such as C++) are impure, and so most OO systems combine big chunks of procedural (i.e. imperative) code. Many people who think they’re doing OOP are actually packaging up procedures.</p><p>Note that I've tried not to list multi-paradigm languages such as Ada, C++ and F# in the above categorisation. It's possible that some of the languages I've listed are also multi-paradigm, but anyway.</p><p>One other way to think about languages is whether they’re top-down or bottom-up: </p><p>Bottom-up languages are ultimately layered on how a processor works (from machine code to assembly language to C & C++), while top-down languages start from the world of mathematics and logic and add language features that allow them to be used for programming (i.e. declarative languages are therefore top-down). This latter set of languages are starting to see increased adoption, as they assume much less (even nothing) about the underlying machinery, in which big changes are occurring with multiple processing cores being introduced (which essentially invalidate the assumptions of previous generations of programmers, who have been conditioned to think in terms of the processor's ability to store and access state).</p><p>Many popular - or soon to be popular - programming environments are pragmatic in nature: C++ allows OOP but can also be used for procedural programming, VB.NET now allows you to define and access objects while coming from a long line of procedural languages, F# is multi-paradigm, combining OO with functional and imperative programming. </p><p>There are bound to be people with differing views on this subject (and many of them are no doubt more intelligent and experienced in these matters than I), but this is how I would answer the question of how to categorise programming languages.</p><p>For those of you with an interest in the future of programming languages, I can strongly recommend the following Channel 9 episodes. If you're not aware of Channel 9, then prepare to be impressed: Microsoft has given a fantastic gift to the development community with this resource.</p><p><a href="http://channel9.msdn.com/Showpost.aspx?postid=382639"><font size="2">Burton Smith: On General Purpose Super Computing and the History and Future of Parallelism</font></a><br/><a href="http://channel9.msdn.com/Showpost.aspx?postid=374141"><font size="2">Erik Meijer: Functional Programming</font></a><br/><a href="http://channel9.msdn.com/Showpost.aspx?postid=273697"><font size="2">Anders Hejlsberg, Herb Sutter, Erik Meijer, Brian Beckman: Software Composability and the Future of Languages</font></a><br/><a href="http://channel9.msdn.com/Showpost.aspx?postid=358968"><font size="2">Brian Beckman: Don't fear the Monads</font></a><br/><a href="http://channel9.msdn.com/showpost.aspx?postid=351659"><font size="2">Joe Armstrong - On Erlang, OO, Concurrency, Shared State and the Future, Part 1</font></a><br/><a href="http://channel9.msdn.com/Showpost.aspx?postid=352136"><font size="2">Joe Armstrong - On Erlang, OO, Concurrency, Shared State and the Future, Part 2</font></a></p><p>Enjoy! :-)<br/></p> <p>四、销毁AutoCAD对象的时机和方法</p><p>June 16, 2008<br/>Cleaning up after yourself: how and when to dispose of AutoCAD objects in .NET<br/>A question came up recently in an internal discussion and I thought I'd share it as it proved so illuminating.</p><p>If I have an object of a type which implements IDisposable, is it good practice to explicitly dispose it (whether via the using statement or calling Dispose() explicitly)?</p><p>The quick(ish) answer is:</p><p>Yes it is, but sometimes you might choose not to as the increase in code simplicity outweighs the benefits derived from manually disposing of the objects.</p><p>So, naturally, the devil is in the detail. Let's take a look at the three scenarios where you're likely to be working with IDisposable objects inside AutoCAD:</p><p>Temporary objects - such as those provided by Autodesk.AutoCAD.Geometry - which are never Database-resident <br/>Temporary objects with the potential to be database-resident but which never actually get added to a Database <br/>Database-resident objects added/accessed via a Transaction<br/>Below follows the details on each of these categories.</p><p>Temporary objects of types not derived from DBObject<br/>The first category of temporary objects, such as Geometry.Line, are safe to be disposed of either "manually" (by your own code) or "automatically" (by the .NET garbage collector).</p><p>Temporary objects of DBObject-derived types<br/>The second category of temporary objects, which are of a type derived from DBObject, must be disposed of manually. It is absolutely unsafe not to dispose of objects of DBObject-derived classes from the main execution thread in AutoCAD.</p><p>Why is this the case? Firstly, the majority of the classes available through AutoCAD's .NET API are currently unmanaged, with a relatively thin wrapper exposing them to the .NET world. Inside AutoCAD, all Database-resident objects are managed by a single-threaded runtime component, AcDb (which, along with some other components, is productized as Autodesk RealDWG). A side note: if you're using ObjectARX or RealDWG from C++, don't be confused by the fact your project's C-runtime memory management is likely to be "Multi-threaded DLL", RealDWG is not thread-aware and so must be treated as single-threaded, for all intents and purposes.</p><p>And - secondly - on to the reason that automatic garbage collection is not to be trusted on DBObject-derived types: the .NET garbage collector runs on a separate, typically low-priority - unless memory is running short - thread. So if a DBObject-derived type is garbage-collected inside AutoCAD, a separate thread will essentially call into a non thread-safe component (RealDWG), which is very, very likely to cause AutoCAD to crash.</p><p>So you must always call Dispose() on temporary objects of DBObject-derived classes in your .NET code - you cannot rely on the CLR to manage your objects' lifetimes for you.</p><p>Interestingly, this is also the reason why the F# code I posted in this prior post will not effectively leverage multiple cores (something I've just tested since getting a multi-core machine). We are using a Ray (which is a DBObject-derived class) to get intersection points with our path, and then disposing of this temporary object from an arbitrary thread (farmed off via F# Asynchronous Workflows). So this is unsafe, and won't run for long before crashing. At least now I understand why.</p><p>Database-resident, Transaction-managed objects<br/>The third category of objects are those we use a Transaction either to add to the Database or to open for access. The good news is that - as the Transaction is aware of the objects it manages (via calls to AddNewlyCreatedDBObject() or to GetObject()), it is able to dispose of them automatically when it, itself, is disposed. So the key here is to make sure you wrap your use of Transactions in using blocks or call Dispose() on them when you're done. There is no need to explicitly dispose of the objects managed by a Transaction (unless there is a failure between the time the object is created and when it is added to the transaction via AddNewlyCreatedDBObject(), of course).</p><p></p> <p>五、</p><p>October 27, 2008</p><p>AU Handouts: AutoCAD® .NET - Developing for AutoCAD® Using F#</p><p><a href="http://through-the-interface.typepad.com/through_the_interface/2008/10/au-handouts-aut.html">http://through-the-interface.typepad.com/through_the_interface/2008/10/au-handouts-aut.html</a></p><p><a href="http://through-the-interface.typepad.com/through_the_interface/2008/10/au-handouts-a-1.html">http://through-the-interface.typepad.com/through_the_interface/2008/10/au-handouts-a-1.html</a></p> 六、一个简单的注标工具(?)
January 28, 2009
Implementing a simple graphing tool inside AutoCAD using F#
Well, I couldn't resist... as I mentioned in the last post - where we looked at creating a simple graph inside AutoCAD as an example of modifying objects inside nested transactions - the idea of graphing inside AutoCAD is a good fit for F#. This is for a number of reasons: F# is very mathematical in nature and excels at processing lists of data. I also spiced it up a bit by adding some code to parallelise some of the mathematical operations, but that didn't turn out to be especially compelling with my dual-core laptop. More on that later.
Here's the F# code:
// Use lightweight F# syntax
#light
// Declare a specific namespace and module name
module Grapher.Commands
// Import managed assemblies
open Autodesk.AutoCAD.Runtime
open Autodesk.AutoCAD.ApplicationServices
open Autodesk.AutoCAD.DatabaseServices
open Autodesk.AutoCAD.Geometry
// Define a common normalization function which makes sure
// our graph gets mapped to our grid
let normalize fn normFn x minInp maxInp maxOut =
let res =
fn ((maxInp - minInp) * x / maxOut)
let normRes = normFn res
if normRes >= 0.0 && normRes <= 1.0 then
normRes * (maxOut - 1.0)
else
-1.0
// Define some shortcuts to the .NET Math library
// trigonometry functions
let sin x = System.Math.Sin x
let cos x = System.Math.Cos x
let tan x = System.Math.Tan x
// Implement our own normalized trig functions
// which each map to the size of the grid passed in
let normSin max x =
let nf a = (a + 1.0) / 2.0 // Normalise to 0-1
let res =
normalize
sin nf (Int32.to_float x)
0.0 (2.0 * System.Math.PI) (Int32.to_float max)
Int32.of_float res
let normCos max x =
let nf a = (a + 1.0) / 2.0 // Normalise to 0-1
let res =
normalize
cos nf (Int32.to_float x)
0.0 (2.0 * System.Math.PI) (Int32.to_float max)
Int32.of_float res
let normTan max x =
let nf a = (a + 3.0) / 6.0 // Normalise differently for tan
let res =
normalize
tan nf (Int32.to_float x)
0.0 (2.0 * System.Math.PI) (Int32.to_float max)
Int32.of_float res
// Now we declare our command
[<CommandMethod("graph")>]
let gridCommand() =
// We'll time the command, so we can check the
// sync vs. async efficiency
let starttime = System.DateTime.Now
// Let's get the usual helpful AutoCAD objects
let doc =
Application.DocumentManager.MdiActiveDocument
let ed = doc.Editor
let db = doc.Database
// "use" has the same effect as "using" in C#
use tr =
db.TransactionManager.StartTransaction()
// Get appropriately-typed BlockTable and BTRs
let bt =
tr.GetObject
(db.BlockTableId,OpenMode.ForRead)
:?> BlockTable
let ms =
tr.GetObject
(bt.,
OpenMode.ForWrite)
:?> BlockTableRecord
// Function to create a filled circle (hatch) at a
// specific location
// Note the valid use of tr and ms, as they are in scope
let createCircle pt rad =
let hat = new Hatch()
hat.SetDatabaseDefaults();
hat.SetHatchPattern
(HatchPatternType.PreDefined,
"SOLID")
let id = ms.AppendEntity(hat)
tr.AddNewlyCreatedDBObject(hat, true)
// Now we create the loop, which we make db-resident
// (appending a transient loop caused problems, so
// we're going to use the circle and then erase it)
let cir = new Circle()
cir.Radius <- rad
cir.Center <- pt
let lid = ms.AppendEntity(cir)
tr.AddNewlyCreatedDBObject(cir, true)
// Have the hatch use the loop we created
let loops = new ObjectIdCollection()
loops.Add(lid) |> ignore
hat.AppendLoop(HatchLoopTypes.Default, loops)
hat.EvaluateHatch(true)
// Now we erase the loop
cir.Erase()
id
// Function to create our grid of circles
let createGrid size rad offset =
let ids = new ObjectIdCollection()
for i = 0 to size - 1 do
for j = 0 to size - 1 do
let pt =
new Point3d
(offset * (Int32.to_float i),
offset * (Int32.to_float j),
0.0)
let id = createCircle pt rad
ids.Add(id) |> ignore
ids
// Function to change the colour of an entity
let changeColour col (id : ObjectId) =
if id.IsValid then
let ent =
tr.GetObject(id, OpenMode.ForWrite) :?> Entity
ent.ColorIndex <- col
// Shortcuts to make objects red and yellow
let makeRed = changeColour 1
let makeYellow = changeColour 2
// Function to retrieve the contents of our
// array of object IDs - this just calculates
// the index based on the x & y values
let getIndex fn size i =
let res = fn size i
if res >= 0 then
(i * size) + res
else
-1
// Apply our function synchronously for each value of x
let applySyncBelowMax size fn =
[| for i in ->
getIndex fn size i |]
// Apply our function asynchronously for each value of x
let applyAsyncBelowMax size fn =
Async.Run
(Async.Parallel
[ for i in ->
async { return getIndex fn size i } ])
// Hardcode the size of the grid and create it
let size = 50
let ids = createGrid size 0.5 1.2
// Make the circles all red to start with
Seq.iter makeRed (Seq.cast ids)
// From a certain index in the list, get an object ID
let getId i =
if i >= 0 then
ids.
else
ObjectId.Null
// Apply one of our trig functions, synchronously or
// otherwise, to our grid
applySyncBelowMax size normSin |>
Array.map getId |>
Array.iter makeYellow
// Commit the transaction
tr.Commit()
// Check how long it took
let elapsed =
System.DateTime.op_Subtraction
(System.DateTime.Now, starttime)
ed.WriteMessage
("\nElapsed time: " +
elapsed.ToString())Here's what you see on AutoCAD's drawing canvas when you run the GRAPH command as it stands:
If you want to play around with other functions, you can edit the call to applySyncBelowMax to pass normCos or normTan instead of normSin.
As I mentioned earlier, if you swap the call to be applyAsyncBelowMax instead of applySyncBelowMax you will actually run the mathematics piece as asynchronous tasks. These are CPU-bound operations - they don't call across the network or write to a hard-drive, which might have increased the benefit of calling them asynchronously - so right now the async version actually runs more slowly than the sync version. If I were to have more processing cores available to me, it might also give us more benefit, but right now with my dual-core machine there's more effort spent coordinating the tasks than you gain from the parallelism. But I'll let you play around with that yourselves... you may get better results. One other note on that piece of the code: at some point I'd like to make use of the Parallel Extensions for .NET (in particular the Task Parallel Library (TPL)), but for now I've continued with what I know, the asynchronous worklows capability which is now standard in F#.
I'm travelling in India this week (and working from our Bangalore office next week), so this is likely to be my last post of the week.
本帖最后由 作者 于 2009-9-9 8:51:21 编辑
七、并行像素化
February 02, 2009
Parallelized pixelization inside AutoCAD using F#
As promised in the last post, we're now going to look at how to change the code to make the colour averaging routine work in parallel. The overall performance is marginally better on my dual-core machine, but I fully expect it to get quicker and quicker as the number of cores multiply.
To start with, though, here's the modified "synchronous" version of the code - as I went through making the code work in parallel, I noticed a bunch of general enhancements that were applicable to both versions. Here's the updated F# code:
// Use lightweight F# syntax
#light
// Declare a specific namespace and module name
module SyncPixelizer.Commands
// Import managed assemblies
#nowarn "9" // ... because we're using NativePtr
open Autodesk.AutoCAD.Runtime
open Autodesk.AutoCAD.ApplicationServices
open Autodesk.AutoCAD.DatabaseServices
open Autodesk.AutoCAD.EditorInput
open Autodesk.AutoCAD.Geometry
open Autodesk.AutoCAD.Colors
open System.Drawing.Imaging
open Microsoft.FSharp.NativeInterop
// Add up the RGB values of a list of pixels
// We use a recursive function with an accumulator argument,
// (rt, gt, bt), to allow tail call optimization
let rec sumColors (pix : List<(byte*byte*byte)>) (rt,gt,bt) =
match pix with
| [] -> (rt, gt, bt)
| (r, g, b) :: tl ->
sumColors tl
(rt + Byte.to_int r,
gt + Byte.to_int g,
bt + Byte.to_int b)
// Average out the RGB values of a list of pixels
let getAverageColour (pixels : List<(byte*byte*byte)>) =
let (rsum, gsum, bsum) =
sumColors pixels (0, 0, 0)
let count = pixels.Length
let ravg = Byte.of_int (rsum / count)
let gavg = Byte.of_int (gsum / count)
let bavg = Byte.of_int (bsum / count)
// For some reason the pixel needs ro be reversed - probably
// because of the bitmap format (needs investigation)
Color.FromRgb(bavg, gavg, ravg)
// Function to get an index into our flat array
// from an x,y pair
let getIndexFromXY ysize x y =
(x * ysize) + y
//Get a chunk of pixels to average from one row
// We use a recursive function with an accumulator argument
// to allow tail call optimization
let rec getChunkRowPixels p xsamp acc =
if xsamp = 0 then
acc
else
let pix =
[(NativePtr.get p 0,
NativePtr.get p 1,
NativePtr.get p 2)]
let p = NativePtr.add p 3 // We do *not* mutate here
getChunkRowPixels p (xsamp-1) (pix @ acc)
// Get a chunk of pixels to average from multiple rows
// We use a recursive function with an accumulator argument
// to allow tail call optimization
let rec getChunkPixels p stride xsamp ysamp acc =
if ysamp = 0 then
acc
else
let pix = getChunkRowPixels p xsamp []
let p = NativePtr.add p stride// We do *not* mutate here
getChunkPixels p stride xsamp (ysamp-1) (pix @ acc)
// Get the various chunks of pixels to average across
// a complete bitmap image
let pixelizeBitmap (image:System.Drawing.Bitmap) xsize ysize =
// Create a 1-dimensional array of pixel lists (one list,
// which then needs averaging, per final pixel)
let (arr : List<(byte*byte*byte)>[]) =
Array.create (xsize * ysize) []
// Lock the entire memory block related to our image
let bd =
image.LockBits
(System.Drawing.Rectangle
(0, 0, image.Width ,image.Height),
ImageLockMode.ReadOnly, image.PixelFormat)
// Establish the number of pixels to sample per chunk
// in each of the x and y directions
let xsamp = image.Width / xsize
let ysamp = image.Height / ysize
// We have a mutable pointer to step through the image
let mutable (p:nativeptr<byte>) =
NativePtr.of_nativeint (bd.Scan0)
// Loop through the various chunks
for i = 0 to ysize - 1 do
// We take a copy of the current value of p, as we
// don't want to mutate p while extracting the pixels
// within a row
let mutable xp = p
for j = 0 to xsize - 1 do
// Get the square chunk of pixels starting at
// this x,y position
let chk =
getChunkPixels xp bd.Stride xsamp ysamp []
// Add it into our array
let idx = getIndexFromXY ysize j (ysize-1-i)
arr. <- chk
// Mutate the pointer to move along to the right
// by a value of 3 (our RGB value) times the
// number of pixels we're sampling in x
xp <- NativePtr.add xp (xsamp * 3)
done
// Mutate the original p pointer to move on one row
p <- NativePtr.add p (bd.Stride * ysamp)
done
// Finally unlock the bitmap data and return the array
image.UnlockBits(bd)
arr
// Create an array of ObjectIds from a collection
let getIdArray (ids : ObjectIdCollection) =
[| for i in -> ids. |]
// Declare our command
[<CommandMethod("pix")>]
let pixelize() =
// Let's get the usual helpful AutoCAD objects
let doc =
Application.DocumentManager.MdiActiveDocument
let ed = doc.Editor
let db = doc.Database
// Prompt the user for the file and the width of the image
let pofo =
new PromptOpenFileOptions
("Select an image to import and pixelize")
pofo.Filter <-
"Jpeg Image (*.jpg)|*.jpg|All files (*.*)|*.*"
let pfnr = ed.GetFileNameForOpen(pofo)
let file =
match pfnr.Status with
| PromptStatus.OK ->
pfnr.StringResult
| _ ->
""
if System.IO.File.Exists(file) then
let img = System.Drawing.Image.FromFile(file)
let pio =
new PromptIntegerOptions
("\nEnter number of horizontal pixels: ")
pio.AllowNone <- true
pio.UseDefaultValue <- true
pio.LowerLimit <- 1
pio.UpperLimit <- img.Width
pio.DefaultValue <- 100
let pir = ed.GetInteger(pio)
let xsize =
match pir.Status with
| PromptStatus.None ->
img.Width
| PromptStatus.OK ->
pir.Value
| _ -> -1
if xsize > 0 then
// Calculate the vertical size from the horizontal
let ysize = img.Height * xsize / img.Width
if ysize > 0 then
// We'll time the command, so we can check the
// sync vs. async efficiency
let starttime = System.DateTime.Now
// "use" has the same effect as "using" in C#
use tr =
db.TransactionManager.StartTransaction()
// Get appropriately-typed BlockTable and BTRs
let bt =
tr.GetObject
(db.BlockTableId,OpenMode.ForRead)
:?> BlockTable
let ms =
tr.GetObject
(bt.,
OpenMode.ForWrite)
:?> BlockTableRecord
// Function to create a filled circle (hatch) at a
// specific location
// Note the valid use of tr and ms, as they are in scope
let createCircle pt rad =
let hat = new Hatch()
hat.SetDatabaseDefaults()
hat.SetHatchPattern
(HatchPatternType.PreDefined,
"SOLID")
let id = ms.AppendEntity(hat)
tr.AddNewlyCreatedDBObject(hat, true)
// Now we create the loop, which we make db-resident
// (appending a transient loop caused problems, so
// we're going to use the circle and then erase it)
let cir = new Circle()
cir.Radius <- rad
cir.Center <- pt
let lid = ms.AppendEntity(cir)
tr.AddNewlyCreatedDBObject(cir, true)
// Have the hatch use the loop we created
let loops = new ObjectIdCollection()
loops.Add(lid) |> ignore
hat.AppendLoop(HatchLoopTypes.Default, loops)
hat.EvaluateHatch(true)
// Now we erase the loop
cir.Erase()
id
// Function to create our grid of circles
let createGrid xsize ysize rad offset =
let ids = new ObjectIdCollection()
for i = 0 to xsize - 1 do
for j = 0 to ysize - 1 do
let pt =
new Point3d
(offset * (Int32.to_float i),
offset * (Int32.to_float j),
0.0)
let id = createCircle pt rad
ids.Add(id) |> ignore
ids
// Function to change the colour of an entity
let changeColour (id : ObjectId) (col : Color) =
if id.IsValid then
let ent =
tr.GetObject(id, OpenMode.ForWrite) :?> Entity
ent.Color <- col
// Create our basic grid
let ids = createGrid xsize ysize 0.5 1.2
// Cast our image to a bitmap and then
// get the chunked pixels
let bmp = img :?> System.Drawing.Bitmap
let arr = pixelizeBitmap bmp xsize ysize
// Loop through the pixel list and average them out
// (which could be parallelized), using the results
// to change the colour of the circles in our grid
Array.map getAverageColour arr |>
Array.iter2 changeColour (getIdArray ids)
// Commit the transaction
tr.Commit()
// Check how long it took
let elapsed =
System.DateTime.op_Subtraction
(System.DateTime.Now, starttime)
ed.WriteMessage
("\nElapsed time: " + elapsed.ToString())
To change this to run the colour averaging asynchronously (in parallel, if you have the cores) is really simple. We replace one line of code "Array.map getAverageColour arr" with the following:
Async.Run
(Async.Parallel
[ for a in arr ->
async { return getAverageColour a }])
This essentially performs a parallel array map (albeit a somewhat naive one), returning basically the same results as the previous line - just hopefully a little more quickly. In case you want to build the two files into one project to test them side-by-side, here they are, the synchronous and asynchronous versions, with the changes needed to allow them to live in and execute from the same assembly.
Here's one more image that's been processed by the PIX command:
In case you're interested, the original image can be found here.
页:
[1]