ShadowTeq.com, the new home of EverHarvest, is officially online! New purchases and license extensions are still currently disabled, but I hope to rectify that situation very soon. The new EverHarvest launcher is available from the site, so I can once again provide updates myself, without any of the lame manual downloading that was necessary over the holiday season.
At present I don't plan to use forums for customer support; I'll be using a combination between email support, a knowledge base, and comments on knowledge base articles to provide help and support. The infrastructure for this is currently online, but I haven't written the articles yet. These will come very soon as I start converting such things as the original EverHarvest guide, and some of the common problems encountered from the old forums. In the meantime, if you require support, email me directly from the email address listed on the ShadowTeq site.
Wednesday, December 30, 2009
Thursday, December 17, 2009
Life Update
So almost two months ago, I found a new job.
After I came back home from Seattle, my wife and I stayed at her dad's house - he helped us to get back home, provided a place to stay for us and our children, on the expectation that it would take me around 2 months (in the current economy) to find a job. I looked daily - I started out with a narrow focus, looking for software development jobs so that I could continue doing what I love to do. During this period I had two telephone interviews for Blackbaud, and one for BenefitFocus; both of which considered my lack of team development experience as a reason not to hire me. As the weeks ticked by, I broadened my scope - networking jobs, PC repair jobs. After my first jobless month I began looking into contracting work, and had some small successes there with another contact of mine in NYC - but it wasn't stable or steady enough to earn me a living.
I began to lose hope in finding a job in my field. At around the second month I began looking locally at jobs in computer retail outlets, where at least I could use my knowledge to help me sell stuff. None of the local houses were hiring, and neither were the chains - Best Buy, Circuit City, etc. I began to consider retail in any form - the local mall, other places - but none were hiring (or, at least, hiring someone with no retail experience).
During the fourth month I began to scrape the bottom of the barrel. Gas stations, fast-food, Target or Wal-Mart. I had a fairly promising interview with a telemarketing company, which might have become my fate if not for Tasha's diligence - you see, she never gave up looking for programming jobs in the first place. Throughout this whole process she would send me a list of programming jobs to apply for - and although I had already given up hope of getting one, she would hound me until I applied anyway.
Thank God for Tasha.
Near the end of the fourth month - after Ta's dad had told us that he wanted us out, kids and all - I got an email from a corporate recruiter for a multi-national military contractor who was interested in interviewing me. I wasn't terribly excited - I figured it would be another dead end, but I called her anyway. She asked me the standard bits - questions about my experience, schooling, etc. After we were done she asked me to come in for an in-person - I of course accepted, but still wasn't optimistic. The in-person is usually where I bomb; I get nervous, and when I'm nervous I sound like I have no idea what I'm talking about.
But there was one plus-side: she told me that my interviewer wanted to see a code review. Here at least I would have the chance to prove I'm capable of programming, even if I stammer like an idiot through the interview. So that's what I did - specifically, I spent about 30 minutes building a simple recursive descent expression parser and evaluator in C# (very similar to the one I wrote in F#), and send that on. It ended up around 150 lines or so - 3 printed pages, so I figured it was large enough to serve as a sample, yet small enough to evaluate.
The interview day came, and off I went. I met my future boss (Dan) and his senior developer, and had the interview. We discussed my resume, my experience, my schooling. Soon we came to the code sample - which Dan had printed out and brought with him. At this point I was still quite nervous, though Dan had a very open demeanor - but when the code sample came up, I knew that this would be the thing to either make me or break me, as it were.
I shouldn't have worried so much - "Quite pleased" I believe the words were, describing the code sample I submitted. He asked me why I picked that particular topic, if I'd used that piece of code in a project - I told him no, but I've always been fascinated with compilers and languages, so since I wasn't given a specific assignment I figured I'd display that interest in the code sample. The more we talked about the code sample, the more they seemed to like me, and the less nervous I became.
After that we talked about technology, the trends therein, and how closely I follow new developments, and that sort of thing. I asked about the team, who I'd be working with, and where I'd fall in the spectrum. Surprisingly, he told me that the entire team is largely made up of lone coders, like me, who, also like me, are looking for more than the lone coder experience - they, like me, want to build a cohesive team that works together, rather than as separate entities on separate projects.
I was to suffer for a week before I found out whether I got the job or not. After the interview, I was stoked - this was a company that seemed to really get it, and was trying to put together a solid development team. I badly wanted this job - not just because I needed it, but because I truly wanted it.
Needless to say, I got the job. =) So this is where I work - and so far, it's been a blast. =) The people are great, the work is stimulating, and the company is stable - growing, in fact, despite the economy. I joined sort of at the ground floor on this particular team, and already I'm helping to set policy and precedent regarding how the team will work. I'm part of a committee tasked with developing a team-wide code standard, and along with our senior developer I'm helping to bring everyone up to speed on .NET (we have a great variety of development talent and experience, but not much .NET).
So there you have it - that's been my life for the past 7 weeks. =) Things are looking up for me, and despite the fact that I write code all day, my interest in it is actually *growing* rather than shrinking, as it did at my last place of employment.
After I came back home from Seattle, my wife and I stayed at her dad's house - he helped us to get back home, provided a place to stay for us and our children, on the expectation that it would take me around 2 months (in the current economy) to find a job. I looked daily - I started out with a narrow focus, looking for software development jobs so that I could continue doing what I love to do. During this period I had two telephone interviews for Blackbaud, and one for BenefitFocus; both of which considered my lack of team development experience as a reason not to hire me. As the weeks ticked by, I broadened my scope - networking jobs, PC repair jobs. After my first jobless month I began looking into contracting work, and had some small successes there with another contact of mine in NYC - but it wasn't stable or steady enough to earn me a living.
I began to lose hope in finding a job in my field. At around the second month I began looking locally at jobs in computer retail outlets, where at least I could use my knowledge to help me sell stuff. None of the local houses were hiring, and neither were the chains - Best Buy, Circuit City, etc. I began to consider retail in any form - the local mall, other places - but none were hiring (or, at least, hiring someone with no retail experience).
During the fourth month I began to scrape the bottom of the barrel. Gas stations, fast-food, Target or Wal-Mart. I had a fairly promising interview with a telemarketing company, which might have become my fate if not for Tasha's diligence - you see, she never gave up looking for programming jobs in the first place. Throughout this whole process she would send me a list of programming jobs to apply for - and although I had already given up hope of getting one, she would hound me until I applied anyway.
Thank God for Tasha.
Near the end of the fourth month - after Ta's dad had told us that he wanted us out, kids and all - I got an email from a corporate recruiter for a multi-national military contractor who was interested in interviewing me. I wasn't terribly excited - I figured it would be another dead end, but I called her anyway. She asked me the standard bits - questions about my experience, schooling, etc. After we were done she asked me to come in for an in-person - I of course accepted, but still wasn't optimistic. The in-person is usually where I bomb; I get nervous, and when I'm nervous I sound like I have no idea what I'm talking about.
But there was one plus-side: she told me that my interviewer wanted to see a code review. Here at least I would have the chance to prove I'm capable of programming, even if I stammer like an idiot through the interview. So that's what I did - specifically, I spent about 30 minutes building a simple recursive descent expression parser and evaluator in C# (very similar to the one I wrote in F#), and send that on. It ended up around 150 lines or so - 3 printed pages, so I figured it was large enough to serve as a sample, yet small enough to evaluate.
The interview day came, and off I went. I met my future boss (Dan) and his senior developer, and had the interview. We discussed my resume, my experience, my schooling. Soon we came to the code sample - which Dan had printed out and brought with him. At this point I was still quite nervous, though Dan had a very open demeanor - but when the code sample came up, I knew that this would be the thing to either make me or break me, as it were.
I shouldn't have worried so much - "Quite pleased" I believe the words were, describing the code sample I submitted. He asked me why I picked that particular topic, if I'd used that piece of code in a project - I told him no, but I've always been fascinated with compilers and languages, so since I wasn't given a specific assignment I figured I'd display that interest in the code sample. The more we talked about the code sample, the more they seemed to like me, and the less nervous I became.
After that we talked about technology, the trends therein, and how closely I follow new developments, and that sort of thing. I asked about the team, who I'd be working with, and where I'd fall in the spectrum. Surprisingly, he told me that the entire team is largely made up of lone coders, like me, who, also like me, are looking for more than the lone coder experience - they, like me, want to build a cohesive team that works together, rather than as separate entities on separate projects.
I was to suffer for a week before I found out whether I got the job or not. After the interview, I was stoked - this was a company that seemed to really get it, and was trying to put together a solid development team. I badly wanted this job - not just because I needed it, but because I truly wanted it.
Needless to say, I got the job. =) So this is where I work - and so far, it's been a blast. =) The people are great, the work is stimulating, and the company is stable - growing, in fact, despite the economy. I joined sort of at the ground floor on this particular team, and already I'm helping to set policy and precedent regarding how the team will work. I'm part of a committee tasked with developing a team-wide code standard, and along with our senior developer I'm helping to bring everyone up to speed on .NET (we have a great variety of development talent and experience, but not much .NET).
So there you have it - that's been my life for the past 7 weeks. =) Things are looking up for me, and despite the fact that I write code all day, my interest in it is actually *growing* rather than shrinking, as it did at my last place of employment.
Monday, December 14, 2009
The Future of EverHarvest
Michael and I have decided to separate our business - EverHarvest is no longer being hosted on his site or supported via his forums.
This does not mean, however, that I will not be continuing to keep it running. At present, the authentication service that allows users with active licenses will continue to run on Michael's site, and I will be effectively extending licenses indefinitely - that is to say, if you purchased an EverHarvest license at some point in the past, then you will very soon be able to use it regardless of whether your license is expired or not. This is a temporary measure however since license extension purchases have been removed from Michael's site.
I will continue performing updates to EverHarvest that are required as a result of EverQuest patches and client changes. In the event that a patch occurs before I have my new site up and running, I will be working with Michael to publish the changes and maintain EverHarvest's working status.
At the moment I am unable to field support requests except via my email inbox, which you can reach by following the Contact Us link at the top of the site.
Know that Michael made a decision he felt necessary to protect his business, and that he's not 'screwing me' or anything like that. He feels that my inability to provide timely updates to EverHarvest impacted negatively on his business (by virtue of association), and he has taken steps to mitigate any damage to his business's reputation it might cause.
As far as the money goes - Michael and I did share some of the revenue for EverHarvest at one point, but that was part of a deal regarding hosting and support (which he ended up paying back to me at the time anyway). He has provided the hosting of EverHarvest for free for longer than a year (if memory serves). I wrote the Autocrat framework upon which both EverHarvest and EverCraft are built, and he paid me for my time and effort.
Summary: it is my intention to build a new site for hosting EverHarvest. Michael is going to host the authentication server temporariliy while I get the new site up and running. During this time period, there will be no need to purchase license extensions for EverHarvest - your software will continue to work regardless of how much time is left on it. After the new site is up and running, your licenses will be back to normal (and I'll be applying some heavy license extensions as well to cover the last couple of downtimes, plus the time spent in this "limbo" between sites). Michael and I are communicating amicably, and are going in fact considering setting up a sort of 'partnership' whereby we recommend each other's services. There aren't any details on this yet but expect to see something on my new site (still unnamed) within the next couple of weeks.
PLEASE don't go on his forums and try to defend me, ESPECIALLY if your defense includes personal attacks on Michael or his business. That sort of thing will not be condoned by either of us - and while I appreciate the loyalty, it's entirely misdirected.
This does not mean, however, that I will not be continuing to keep it running. At present, the authentication service that allows users with active licenses will continue to run on Michael's site, and I will be effectively extending licenses indefinitely - that is to say, if you purchased an EverHarvest license at some point in the past, then you will very soon be able to use it regardless of whether your license is expired or not. This is a temporary measure however since license extension purchases have been removed from Michael's site.
I will continue performing updates to EverHarvest that are required as a result of EverQuest patches and client changes. In the event that a patch occurs before I have my new site up and running, I will be working with Michael to publish the changes and maintain EverHarvest's working status.
At the moment I am unable to field support requests except via my email inbox, which you can reach by following the Contact Us link at the top of the site.
Know that Michael made a decision he felt necessary to protect his business, and that he's not 'screwing me' or anything like that. He feels that my inability to provide timely updates to EverHarvest impacted negatively on his business (by virtue of association), and he has taken steps to mitigate any damage to his business's reputation it might cause.
As far as the money goes - Michael and I did share some of the revenue for EverHarvest at one point, but that was part of a deal regarding hosting and support (which he ended up paying back to me at the time anyway). He has provided the hosting of EverHarvest for free for longer than a year (if memory serves). I wrote the Autocrat framework upon which both EverHarvest and EverCraft are built, and he paid me for my time and effort.
Summary: it is my intention to build a new site for hosting EverHarvest. Michael is going to host the authentication server temporariliy while I get the new site up and running. During this time period, there will be no need to purchase license extensions for EverHarvest - your software will continue to work regardless of how much time is left on it. After the new site is up and running, your licenses will be back to normal (and I'll be applying some heavy license extensions as well to cover the last couple of downtimes, plus the time spent in this "limbo" between sites). Michael and I are communicating amicably, and are going in fact considering setting up a sort of 'partnership' whereby we recommend each other's services. There aren't any details on this yet but expect to see something on my new site (still unnamed) within the next couple of weeks.
PLEASE don't go on his forums and try to defend me, ESPECIALLY if your defense includes personal attacks on Michael or his business. That sort of thing will not be condoned by either of us - and while I appreciate the loyalty, it's entirely misdirected.
Wednesday, November 25, 2009
I'm back. =)
Well it's been a stressful several months, but I want to return to blogging.
In case anyone has been curious as to where I've been, here's the scoop. I am no longer living in Seattle WA, and have moved back home to Charleston SC. I lost the job in Seattle, and my family used that event to trigger the move. Apparently it's been coming for a while - everyone (myself included) was homesick. What followed was a week-long cross-country drive from Seattle to Charleston, during which we towed every earthly possession we could fit into our SUV and a UHaul trailer attacked to the back, along with our two kids, roommate, and a cat. If I have one life experience that I'd like to recommend to everyone, it's that - driving across the country.
After arriving back home, we ended up living at my father in law's house, in the upstairs guest apartment, and have been there since. Last month, he told us that we had to leave by the middle of November - hence my post earlier about needing a job in a month or we'd be homeless. I was fighting a bit of angst when I wrote that, so perhaps things weren't as dire as that (happy Ta?) - but it was getting bad.
But three weeks ago I started a new job - almost 4 months after we returned home. The economy sucks - but I'm glad I held out. I'm working for a large multi-national contracting company that builds *things* for our country's and our ally's militaries. I'm working primarily as a .NET developer, but so far I've been called upon to work with VBScript, VBA, JavaScript, and others, in addition to .NET. So far it's been a blast - I love the people, the diversity of work, and the general atmosphere. I will admit that they've got me a bit on the cheap - this is technically an entry-level position - but I don't mind at all. It pays the bills well, and affords me the opportunity to show that I'm worth a lot more than they're expecting, and so far, so good. :)
I have to be a bit careful with what I reveal on the blog - which is why I'm not naming my place of business, nor will I be talking about specific projects. I'll still talk about MacroCrafter stuff as the opportunity presents - I'm still doing updates, planning new software, etc.
A lot of what I'll be talking about soon will be process-oriented stuff. The team I'm on (which consists of 7 developers and 2 analysts) is largely made up of lone coders looking to build their team experience (which is exactly what I'm here for as well). I think we're going to be creating our own process based on Agile, which makes me happy as it's something I've been wanting the opportunity to get into for a while. One of the major perks of this situation is that I will be in on the ground floor helping to build these processes. I'll write more about this later as developments occur.
In case anyone has been curious as to where I've been, here's the scoop. I am no longer living in Seattle WA, and have moved back home to Charleston SC. I lost the job in Seattle, and my family used that event to trigger the move. Apparently it's been coming for a while - everyone (myself included) was homesick. What followed was a week-long cross-country drive from Seattle to Charleston, during which we towed every earthly possession we could fit into our SUV and a UHaul trailer attacked to the back, along with our two kids, roommate, and a cat. If I have one life experience that I'd like to recommend to everyone, it's that - driving across the country.
After arriving back home, we ended up living at my father in law's house, in the upstairs guest apartment, and have been there since. Last month, he told us that we had to leave by the middle of November - hence my post earlier about needing a job in a month or we'd be homeless. I was fighting a bit of angst when I wrote that, so perhaps things weren't as dire as that (happy Ta?) - but it was getting bad.
But three weeks ago I started a new job - almost 4 months after we returned home. The economy sucks - but I'm glad I held out. I'm working for a large multi-national contracting company that builds *things* for our country's and our ally's militaries. I'm working primarily as a .NET developer, but so far I've been called upon to work with VBScript, VBA, JavaScript, and others, in addition to .NET. So far it's been a blast - I love the people, the diversity of work, and the general atmosphere. I will admit that they've got me a bit on the cheap - this is technically an entry-level position - but I don't mind at all. It pays the bills well, and affords me the opportunity to show that I'm worth a lot more than they're expecting, and so far, so good. :)
I have to be a bit careful with what I reveal on the blog - which is why I'm not naming my place of business, nor will I be talking about specific projects. I'll still talk about MacroCrafter stuff as the opportunity presents - I'm still doing updates, planning new software, etc.
A lot of what I'll be talking about soon will be process-oriented stuff. The team I'm on (which consists of 7 developers and 2 analysts) is largely made up of lone coders looking to build their team experience (which is exactly what I'm here for as well). I think we're going to be creating our own process based on Agile, which makes me happy as it's something I've been wanting the opportunity to get into for a while. One of the major perks of this situation is that I will be in on the ground floor helping to build these processes. I'll write more about this later as developments occur.
Tuesday, September 15, 2009
Hunter bot, simplified.
Everyone, thank Tasha (my wife). She said something to me the other day that really got me going on the hunter bot, and actually inspired me to write some real code. She said, "Just write it."
I've been spending so much time thinking about the hithertoo's and whyfor's about it that I haven't touched a single line of relevant code. My biggest problem is and has always been a combination of over-thinking and over-planning. Many of my project concepts are huge ideas - my web game that'll never be built is a prime example of this. It's so large that it would require a team of people as dedicated to it as I am in order to actually happen, but that's a pipe dream.
So in following Tasha's advice, I've managed to get a very simple prototype hunter bot working. To do this, I've opted for a far simpler strategy: instead of following a walkpath, the user selects a 'camp spot', and the system brings viable targets to the camp spot to kill them.
I decided to do this for one primary reason: it'll help me get something out the door. Features can always be added later - with the Workflow system I wrote for Autocrat, this is easy as pie, once I've figured out the logic. But for now, I'd like to get something out to everyone whose waited so long for it - and the camp-bot is it. I'll have a beta version available very soon.
I've been spending so much time thinking about the hithertoo's and whyfor's about it that I haven't touched a single line of relevant code. My biggest problem is and has always been a combination of over-thinking and over-planning. Many of my project concepts are huge ideas - my web game that'll never be built is a prime example of this. It's so large that it would require a team of people as dedicated to it as I am in order to actually happen, but that's a pipe dream.
So in following Tasha's advice, I've managed to get a very simple prototype hunter bot working. To do this, I've opted for a far simpler strategy: instead of following a walkpath, the user selects a 'camp spot', and the system brings viable targets to the camp spot to kill them.
I decided to do this for one primary reason: it'll help me get something out the door. Features can always be added later - with the Workflow system I wrote for Autocrat, this is easy as pie, once I've figured out the logic. But for now, I'd like to get something out to everyone whose waited so long for it - and the camp-bot is it. I'll have a beta version available very soon.
Wednesday, September 9, 2009
Lol
Got my first comment spam today! Somebody (more likely a spambot) posted links to EQ2 plat buying sites on my post regarding running EH in a VM.
I wasn't aware my blog was popular enough to get the attention of spam bots... That's sort of flattering, in a weird and annoying way. =)
I wasn't aware my blog was popular enough to get the attention of spam bots... That's sort of flattering, in a weird and annoying way. =)
Thursday, September 3, 2009
Running EverQuest and ShadowTeq software in a Virtual Machine
Running EverQuest and ShadowTeq software in a Virtual Machine
| VMWare Workstation 6.5.3* | http://www.vmware.com/products/ws/ |
| Windows XP SP2 or better (ISO or CD) | |
| DirectX 9.0c | http://www.microsoft.com/Downloads/details.aspx?FamilyID=9226a611-62fe-4f61-aba1-914185249413&displaylang=en |
| Windows Installer 3.1 Redistributable | http://www.microsoft.com/DownLoads/details.aspx?familyid=889482FC-5F56-4A38-B838-DE776FD4138C&displaylang=en |
| .NET 3.5 Redistributable | http://www.microsoft.com/downloads/details.aspx?FamilyId=333325FD-AE52-4E35-B531-508D977D32A6&displaylang=en |
| EverQuest 2 | http://download.station.sony.com/patch/download/eq2/eq2-us.zip |
| ShadowTeq Software (Golem, EverHarvest) | http://shadowteq.com |
* - VMWare is not free, and costs $189 for a single license, though there is a 30-day trial. Please note that I do not promote software piracy.
Introduction:
Described in this document is the method by which you can set up and run EverQuest 2 and ShadowTeq software inside a Virtual Machine. There are several reasons why you might wish to do this. First, if you are running an operating system which ShadowTeq does not support (such as Linux, MacOS, or future versions of Windows), you can run a virtual machine (VM) that runs a supported operating system, such as Windows XP, in order to run our software. Second, running inside a VM means you can use the rest of your computer while our software is running, rather than being locked into keeping the EverQuest 2 window always-on-top.
This process is fairly straightforward, if advanced. None of the steps below requires altering your current installation in any way, except for installing VMWare Workstation. Still, if you are not at least comfortable following a set of advanced instructions, I suggest finding someone who will be willing to help you through this. The good news however is that once your virtual machine is up and running, any changes you make to it do not in any way impact your existing system - so you can feel free to experiment inside the VM as much as you like.
Step 1: Install VMWare Workstation 6.5.3:
After you've obtained a copy of VMWare Workstation, you'll need to install it. Double-click on the installation file to begin the installation process. This is straightforward - there aren't any important options to select, so choose Typical installation.
Step 2: Create a Windows XP Virtual Machine:
Now that VMWare is installed, it's time to create the Windows XP Virtual Machine. I'm using Windows XP Service Pack 2, but SP3 is also confirmed to work as well. To do this, you'll need to have either a physical XP CD in your CD-ROM drive, or have an ISO image of the XP CD you intend to use. When you're prepared, launch VMWare, and go to the File menu, select New, and select Virtual Machine. In the window that appears, select the Custom option and click Next. In the next page, ensure that the Hardware Compatability selection is set to 'Workstation 6.5', then click Next.
The next page allows you to choose where to install XP from. If you have a physical CD in your drive, select the first option button, then click Next. If you have an ISO image of the CD, select the second option button, click the Browse button to tell VMWare where your ISO file is, then click Next. The next page will ask you for your XP CD key, your full name, and an Administrator password. Fill in these details and click Next. The next page will give you an opportunity to name your Virtual Machine. I just left mine as the default, but you're free to use whatever name you wish. Additionally you can change where the Virtual Machine files are stored. By default, there's a 'Virtual Machines' folder in your 'Documents' folder that VMs get stored in. When you're done here, click Next.
The next page that appears will ask you how many processors to expose to the VM. If you have more than one processor available in your machine, choose Two. Click Next. The next page allows you to specify the maximum amount of physical RAM that the VM will be allowed to allocate. Note that this means the VM will consume AT MOST this amount, but probably less. I recommend 1024mb. After setting your desired maximum RAM, click Next. The next page allows you to determine the type of network connection the VM will have. I recommend leaving it at the default of 'Use network address translation (NAT)' as this will simplify configuration later on. Click Next. The next page is for SCSI configuration - nothing needs to change here; click Next.
The next page allows you to configure a hard drive for your VM. In our case, we want to create a new virtual disk, so leave it at the default and click Next. The next page allows you to select the type of virtual drive; this selection is largely unimportant, so choose the default and click Next. The next page allows you to set the maximum size of the virtual disk. I recommend 24gb for everything that will need to be installed, but you can choose more if you like. The other options on this page are fine at their defaults, so after you set the size, click Next. Click Next again to select the default name for the disk.
Now we've come to the end of the Virtual Machine creation; click Finish to have VMWare finalize your choices, power up your machine, and install Windows XP. This process is completely automated (if you've followed the instructions) and, when finished, will leave you at the Windows XP login prompt. To log in, type the Administrator password that you specified earlier in the password box.
If all's gone well, you should be greeted with a Windows XP desktop, and the Windows XP login sound effect. One of the first things you should do here is to increase the screen resolution. Currently it's set to 800x600 - much too small to be useful. To change it, right-click on your desktop and select 'Properties,' then select the Settings tab. Drag the 'Screen Resolution' slider until the resolution is at least 1024x768 - any less than that and the EverHarvest or Golem windows will overlap the game window by far too much.
Step 3: Install Supporting Software:
Now it's time to start installing software in your Virtual Machine. Assuming you've used XP SP2, you'll need to install DirectX 9.0c, Windows Installer 3.1, and the .NET Framework 3.5. You'll find links to these downloads at the top of this document. From inside your Virtual Machine, download the files, then install them in the order listed above. Remember, install them inside your Virtual Machine, not on your physical system! All the default installation options are fine for this software.
Step 4: Install EverQuest 2:
Finally, we can install EverQuest 2. Use the download link at the top of this document to download the EQ2 launcher (again, from inside your VM). Unzip it, then run it, and it'll install the base files necessary for downloading and patching EQ2. You should know the drill here - log in to your Station account, and let the patcher run... This will probably take several hours, but you can do other things while you wait.
Step 5: Install ShadowTeq Software:
Now it's time to get our software running. Using the download links on our home page, download the software you're interested in (again, while inside the VM), and install it according to the usual instructions.
... That's it. Really. =)
Step 6: Run:
This works the same as it does on your physical computer - run the application you're interested in running, and log in. Note that you will probably get a 'No Licenses Found' error - that's okay. Just follow the instructions found on this page and you'll be good to go. Configure the software as usual, and run it as usual - the only difference is that you're running it inside the VM.
Now here's the slick part - the culmination of all your efforts. Once the software is up and running, press Control + Alt - and your mouse will detach from the VM's mouse, and you can minimize VMWare, or tab away from it, or whatever - do anything else you want on your computer. If your system is beefy enough, you can even run another instance of EQ2 on your physical computer - or, even, in another Virtual Machine. Theoretically you can have as many VMs running at once as you want - the only limits of course are CPU power, available RAM, hard disk space, and networking bandwidth.
Step 7: Shutting Down:
Shutting down a Virtual Machine works the same way as shutting down a real computer - go to Start and select Turn Off Computer, then Turn Off. The system will perform its shut-down routine, then turn itself off. To turn the 'computer' back on again, select the VM in the left-most list and press the green 'Start' button in the toolbar on top of the window.
That's it!
No, really, that's it. Now that you've got the hang of this, feel free to experiment around. It's really hard to mess this up - and even when you do, you're just dealing with virtual machines - just reformat it if you do something unrecoverable. For more advanced uses of the VMWare software, you might want to consult their documentation, available from the VMWare website.
Enjoy, and good luck!
Wednesday, September 2, 2009
Late birthday present
One of our users has managed to get EQ2, EverHarvest, and EverCraft working in a VMWare Virtual Machine, and has shared the instructions on how to get it working with me!
Why is this such a great birthday present? Virtual Machines are like full computers running inside your physical computer - which means it gets its own (virtual) keyboard and its own (virtual) mouse. While EverHarvest and EverCraft still require EQ2 to be the topmost window on the computer, this can be true inside the Virtual Machine instance while NOT being true on your physical computer!
This means that you can DO OTHER THINGS on your computer while you run our software! Finally! =P
I'm working on getting the instructions in a more user-friendly form before posting them - which should come in the next day or so. Just wanted to post this to get the pot stirring, so to speak... =)
Why is this such a great birthday present? Virtual Machines are like full computers running inside your physical computer - which means it gets its own (virtual) keyboard and its own (virtual) mouse. While EverHarvest and EverCraft still require EQ2 to be the topmost window on the computer, this can be true inside the Virtual Machine instance while NOT being true on your physical computer!
This means that you can DO OTHER THINGS on your computer while you run our software! Finally! =P
I'm working on getting the instructions in a more user-friendly form before posting them - which should come in the next day or so. Just wanted to post this to get the pot stirring, so to speak... =)
Monday, August 24, 2009
=)
Happy birthday to me,
Happy birthday to me,
Happy birthday dear... meeeee...
Happy birthday to me!
...
I need a life... Lol
Happy birthday to me,
Happy birthday dear... meeeee...
Happy birthday to me!
...
I need a life... Lol
Thursday, August 20, 2009
Lambdas and Anonymous Delegates pt 2
So I did exactly what I suggested to myself, and sent an email to Eric Lippert regarding the inline lambda and anonymous delegate business I blogged about earlier today. He's quick - here's his reply:
He's right - it's not like this is some sort of necessary functionality. On the one hand, it's great that I can get exactly this functionality by casting to a delegate type first, but on the other hand it's extra cruft. All the same, if some inspired or courageous EverDroid user needs this sort of functionality, it's there.
Thanks, Eric! =)
Ah, good question.
You can make this work by casting to a delegate type.
((Func<bool>)()=>M()).Invoke();
((Func<bool>)(delegate (){return M();}).Invoke();
For lambdas, we don't know whether the lambda is to be converted to a delegate or an expression tree. Since an expression tree cannot be invoked, we cannot allow you to invoke it before we know what you intend.
For anonymous methods, yeah, I suppose we could have designed C# 2.0 to allow invoking them directly. But what would the point be? Why say (delegate(){return M();}).Invoke()" when you could just say "M()"? I agree that for reasons of generality and consistency it would be nice to have, but I don't think disallowing this actually disables many scenarios.
He's right - it's not like this is some sort of necessary functionality. On the one hand, it's great that I can get exactly this functionality by casting to a delegate type first, but on the other hand it's extra cruft. All the same, if some inspired or courageous EverDroid user needs this sort of functionality, it's there.
Thanks, Eric! =)
Inline execution of a lambda expression or anonymous delegate
I had an idea I wanted to try to exploit that would make my VB.NET expression a bit more, well, expressive.
The way I built my expression builder, essentially you pass in a valid VB.NET expression and the expression builder sticks it onto the end of a Return statement inside a static method of a new class, which it then compiles with the stock VB.NET code compiler. Then I use reflection to dig out the compiled method and returns a Func<bool> that invokes the method, returning the result.
This works great for its intended purpose, but it unfortunately restricts my expressions to being, well, expressions. Boolean expressions. Which means they can't do a whole lot of processing, or whatever. That's okay - I don't really *need* all that much. But I enjoy a challenge, and I thought to myself 'Self, why don't we try to use an inline lambda!'
Great idea - except you can't INVOKE a lambda inline. Example:
bool x = ( () => true ).Invoke();
Doesn't work. Operator '.' cannot be applied to operand of type 'lambda expression'. 'Fine,' says I, 'I'll use an anonymous delegate!'
bool x = ( delegate { return true; } ).Invoke();
Operator '.' cannot be applied to operand of type 'anonymous delegate'. Argh. So as far as i'm aware, this is impossible, probably by design. I'll have to ask Eric Lippert about it, since he's been doing his C# design rationale posts lately.
The way I built my expression builder, essentially you pass in a valid VB.NET expression and the expression builder sticks it onto the end of a Return statement inside a static method of a new class, which it then compiles with the stock VB.NET code compiler. Then I use reflection to dig out the compiled method and returns a Func<bool> that invokes the method, returning the result.
This works great for its intended purpose, but it unfortunately restricts my expressions to being, well, expressions. Boolean expressions. Which means they can't do a whole lot of processing, or whatever. That's okay - I don't really *need* all that much. But I enjoy a challenge, and I thought to myself 'Self, why don't we try to use an inline lambda!'
Great idea - except you can't INVOKE a lambda inline. Example:
bool x = ( () => true ).Invoke();
Doesn't work. Operator '.' cannot be applied to operand of type 'lambda expression'. 'Fine,' says I, 'I'll use an anonymous delegate!'
bool x = ( delegate { return true; } ).Invoke();
Operator '.' cannot be applied to operand of type 'anonymous delegate'. Argh. So as far as i'm aware, this is impossible, probably by design. I'll have to ask Eric Lippert about it, since he's been doing his C# design rationale posts lately.
Progress thus far
Truth be told, at this point, progress is slow going. I've been doing a lot of work for my friend Brian, and as it pays more immediately than Macrocrafter work I have to focus on that. It's been calming down of late (unfortunately) so that should afford me more time to work on this stuff.
I've been slowly organizing the work I need to do on this system. So far I've divided it up into several main components that I'll need to build before I can start stitching them together.
Also I've decided to skip the intermediate step and just make a damned hunter bot out of this, so I'll also need to integrate targeting and walking into the system.
Anyway - one of those systems includes a VB.NET compiler, as I mentioned before. All I need to do with this is to take my proof of concept code and make a proper system out of it - something reusable. I'm going to add it to Autocrat. I'll also need a timer that can produce timing events in 150ms intervals - that's the interval at which I'm going to have the system pull down updated EQ2 data. Any of the .NET timers should be capable of doing this so I'm not too worried here. I'm also going to need a state machine engine that can be configured via XML - this will be responsible for persisting the user-editable 'scripts' that will make up the brains of the bot itself. I've got a schema all figured up; all that remains is actually putting the work into it - and then I get to build a UI on top of it, because the last thing I want to require is for people (or myself) to have to hand-edit XML to get it to work.
Coupled with the timer is going to be a system for describing stateful property changes. Here's the problem: my bot engine is going to be evaluating event trigger expressions every 150ms. The expressions are boolean, so they'll be testing data based on the content of the current tick's EQ2 properties. If an expression looks similar to: "Return EQ2.GroupMember(2).Health < 50" - then when group member 2's health drops below 50%, it will trigger the event. This is desired - BUT it will ALSO trigger once every 150ms tick until group member 2's health rises to or above 50. This is NOT desired, so I need a way to expose the concept of state changes: for example, I'd like to rewrite the same expression as: "Return EQ2.GroupMember(2).Health.DroppedBelow(50)". That would return True if, and only if, the previous tick's Health was at or above 50 AND the current tick's Health is below 50.
So I'll need to figure out an efficient system of encapsulating old state and exposing it via these state descriptors. I'm sure I'll figure it out.
But you can see why I've been making slow progress here. =) In between kids, contract work, looking for full-time work, EQ2 updates, and other life, I don't have much time for experimentation, which is a shame.
I've been slowly organizing the work I need to do on this system. So far I've divided it up into several main components that I'll need to build before I can start stitching them together.
Also I've decided to skip the intermediate step and just make a damned hunter bot out of this, so I'll also need to integrate targeting and walking into the system.
Anyway - one of those systems includes a VB.NET compiler, as I mentioned before. All I need to do with this is to take my proof of concept code and make a proper system out of it - something reusable. I'm going to add it to Autocrat. I'll also need a timer that can produce timing events in 150ms intervals - that's the interval at which I'm going to have the system pull down updated EQ2 data. Any of the .NET timers should be capable of doing this so I'm not too worried here. I'm also going to need a state machine engine that can be configured via XML - this will be responsible for persisting the user-editable 'scripts' that will make up the brains of the bot itself. I've got a schema all figured up; all that remains is actually putting the work into it - and then I get to build a UI on top of it, because the last thing I want to require is for people (or myself) to have to hand-edit XML to get it to work.
Coupled with the timer is going to be a system for describing stateful property changes. Here's the problem: my bot engine is going to be evaluating event trigger expressions every 150ms. The expressions are boolean, so they'll be testing data based on the content of the current tick's EQ2 properties. If an expression looks similar to: "Return EQ2.GroupMember(2).Health < 50" - then when group member 2's health drops below 50%, it will trigger the event. This is desired - BUT it will ALSO trigger once every 150ms tick until group member 2's health rises to or above 50. This is NOT desired, so I need a way to expose the concept of state changes: for example, I'd like to rewrite the same expression as: "Return EQ2.GroupMember(2).Health.DroppedBelow(50)". That would return True if, and only if, the previous tick's Health was at or above 50 AND the current tick's Health is below 50.
So I'll need to figure out an efficient system of encapsulating old state and exposing it via these state descriptors. I'm sure I'll figure it out.
But you can see why I've been making slow progress here. =) In between kids, contract work, looking for full-time work, EQ2 updates, and other life, I don't have much time for experimentation, which is a shame.
Saturday, August 1, 2009
Happy Birthday baby girl =) And some other stuff.
So my daughter just turned 2 years old today. We had a party at CiCi's, and fun, pizza, and cake was had all around. =)
A friend of mine in NYC has a lot of extra work he's unable to do himself, due to having his own full-time job, family, etc., so he's farming it out to me. It's a great situation as I currently have a work deficit, so taking it off his hands helps me out a lot. So far I've updated three websites owned by the same company - took me the better part of a day to get it all completed, but it wasn't terribly hard. Mostly grunt-work, but it pays, and it's not brainless. I am by no means complaining. =)
Finding work in Charleston, SC has been difficult - the two large development companies out here have both rejected me as being "not quite what they want" - I don't have any team-oriented experience, or large enterprise experience, which is all they're looking for out here. My argument is that if everyone wants to hire people with prior team experience, how can anyone who doesn't have it get it? Eventually there'll be nobody left except people *without* team experience, and then what will they do?
Bah. Perhaps I'm bitter - I need a job so I can put food on the table and pay rent and all that rot. It's frustrating. I'm actually hoping I find a computer / network tech job I can do, rather than programming - I think I'll have more fun doing that, since I'll get to work with people, help solve their immediate problems, etc.
Maybe I'll see if I can find investors for that computer recycling business I've been thinking of starting.
Anyway - very little work has occurred on my next EQ2 project, which one of my friends has dubbed EverDroid - a name I think I'll use. I need to build an Eval system into Autocrat, because this is something I'm going to use a lot in projects to come, for the sake of configuration and behavior customization. That won't be hard - I've got a simple console proof of concept working just fine, compiling expressions in VB.NET (which I think non-programmers will find easier to reason about - Microsoft agrees, and has done the same thing for their expression compiler in Windows Workflow 4).
EverHarvest had a good month last month - decent sales, despite the economic issues. Not nearly what it used to be, but then again I never expected it to be. Most people seem to be happy with it, when it works - which it's not at the moment; I've been working on the next update in response to Sony's latest changes, but they're doing their rapid-fire patching again, which always makes me want to wait... I was half-way through the last set of updates when they patched again yesterday, ruining my work. Ah well - such is the price to pay. Overall it's nothing to complain about - I work on it maybe two days total out of every month.
So, that's my update today. I'll get back to the millstone again and crank out the EH update. I really need to write some helper tools for this thing...
A friend of mine in NYC has a lot of extra work he's unable to do himself, due to having his own full-time job, family, etc., so he's farming it out to me. It's a great situation as I currently have a work deficit, so taking it off his hands helps me out a lot. So far I've updated three websites owned by the same company - took me the better part of a day to get it all completed, but it wasn't terribly hard. Mostly grunt-work, but it pays, and it's not brainless. I am by no means complaining. =)
Finding work in Charleston, SC has been difficult - the two large development companies out here have both rejected me as being "not quite what they want" - I don't have any team-oriented experience, or large enterprise experience, which is all they're looking for out here. My argument is that if everyone wants to hire people with prior team experience, how can anyone who doesn't have it get it? Eventually there'll be nobody left except people *without* team experience, and then what will they do?
Bah. Perhaps I'm bitter - I need a job so I can put food on the table and pay rent and all that rot. It's frustrating. I'm actually hoping I find a computer / network tech job I can do, rather than programming - I think I'll have more fun doing that, since I'll get to work with people, help solve their immediate problems, etc.
Maybe I'll see if I can find investors for that computer recycling business I've been thinking of starting.
Anyway - very little work has occurred on my next EQ2 project, which one of my friends has dubbed EverDroid - a name I think I'll use. I need to build an Eval system into Autocrat, because this is something I'm going to use a lot in projects to come, for the sake of configuration and behavior customization. That won't be hard - I've got a simple console proof of concept working just fine, compiling expressions in VB.NET (which I think non-programmers will find easier to reason about - Microsoft agrees, and has done the same thing for their expression compiler in Windows Workflow 4).
EverHarvest had a good month last month - decent sales, despite the economic issues. Not nearly what it used to be, but then again I never expected it to be. Most people seem to be happy with it, when it works - which it's not at the moment; I've been working on the next update in response to Sony's latest changes, but they're doing their rapid-fire patching again, which always makes me want to wait... I was half-way through the last set of updates when they patched again yesterday, ruining my work. Ah well - such is the price to pay. Overall it's nothing to complain about - I work on it maybe two days total out of every month.
So, that's my update today. I'll get back to the millstone again and crank out the EH update. I really need to write some helper tools for this thing...
Saturday, July 18, 2009
So far, so good.
Early proofs of concept indicate this might be a viable option. Here are some of the hurdles ahead of me:
Trigger condition evaluation: I'm planning on using VB.NET as the language for trigger condition expressions. I've built a simple test that takes an arbitrary boolean expression, compiles it via CodeDOM into an in-memory assembly, and returns a delegate to invoke it with all the necessary information that will be required of the bot. This works well so far, but of course it remains to be seen how well it'll work in practice. I may change this to having the expression compiled into a System.Linq.Expression, which I could then compile into a delegate - I think that would be faster than the CodeDOM method I'm using now. Profiling will tell.
Dynamic state machines: Part of the system will be handling different states. I'm using states as a logical grouping of events for a common scenario. For example, a Fighter bot could have many states that it could be in at any one time; an out of combat state, an in-combat state, a dead state, and so on. Each of these states can have their own grouping of events, possibly having actions that force the machine to transition between states. I have some experience with state machines so this shouldn't be too much of a challenge.
Ability mapping and timing: Another vital part of the system is my Ability Mapping concept. Instead of having each user modify their scripts to include their character's own personal spells and combat arts, scripts will instead reference ability mappings. These mappings are then configured by the user to refer to one or more in-game abilities, each of which has its own cast, reuse, and duration timers. When a script action invokes the mapping, the system will use the first ability it's able to, according to its position in the mapping (top to bottom) and the status of its timers. This way, said Fighter bot can have a mapping called 'attack' that its user could have configured with all his character's single-target combat arts, so that every time that 'attack' mapping is invoked, one of those combat arts is used. I'm still conceptualizing this particular feature so any more detail would be speculation / brainstorming, which I will save for another post.
Configuration and UI: Of course, all of this complexity needs to be presented as simply as possible. I do not intend for people to manually edit XML in order to change these scripts, so I'll be building a script editing UI for that purpose. In addition, the UI will need to include a mapping editor, as well as a place for entering script-declared variables. All of these things will need to be data-aware, and some of them dynamically created. I've been debating with myself whether I want to use XAML and WPF for the UI, and while this sounds like it might be a good use of WPF, I still haven't taken the time to crest the learning wall, and with my current job situation I don't know if I want to take that time now.
There's more, but this is getting long already - I'll post again either tomorrow or the next day with more.
Trigger condition evaluation: I'm planning on using VB.NET as the language for trigger condition expressions. I've built a simple test that takes an arbitrary boolean expression, compiles it via CodeDOM into an in-memory assembly, and returns a delegate to invoke it with all the necessary information that will be required of the bot. This works well so far, but of course it remains to be seen how well it'll work in practice. I may change this to having the expression compiled into a System.Linq.Expression, which I could then compile into a delegate - I think that would be faster than the CodeDOM method I'm using now. Profiling will tell.
Dynamic state machines: Part of the system will be handling different states. I'm using states as a logical grouping of events for a common scenario. For example, a Fighter bot could have many states that it could be in at any one time; an out of combat state, an in-combat state, a dead state, and so on. Each of these states can have their own grouping of events, possibly having actions that force the machine to transition between states. I have some experience with state machines so this shouldn't be too much of a challenge.
Ability mapping and timing: Another vital part of the system is my Ability Mapping concept. Instead of having each user modify their scripts to include their character's own personal spells and combat arts, scripts will instead reference ability mappings. These mappings are then configured by the user to refer to one or more in-game abilities, each of which has its own cast, reuse, and duration timers. When a script action invokes the mapping, the system will use the first ability it's able to, according to its position in the mapping (top to bottom) and the status of its timers. This way, said Fighter bot can have a mapping called 'attack' that its user could have configured with all his character's single-target combat arts, so that every time that 'attack' mapping is invoked, one of those combat arts is used. I'm still conceptualizing this particular feature so any more detail would be speculation / brainstorming, which I will save for another post.
Configuration and UI: Of course, all of this complexity needs to be presented as simply as possible. I do not intend for people to manually edit XML in order to change these scripts, so I'll be building a script editing UI for that purpose. In addition, the UI will need to include a mapping editor, as well as a place for entering script-declared variables. All of these things will need to be data-aware, and some of them dynamically created. I've been debating with myself whether I want to use XAML and WPF for the UI, and while this sounds like it might be a good use of WPF, I still haven't taken the time to crest the learning wall, and with my current job situation I don't know if I want to take that time now.
There's more, but this is getting long already - I'll post again either tomorrow or the next day with more.
Tuesday, July 14, 2009
Big plans.
So I hinted, last time, about some big plans I have for a new EQ2 project. I think I may have gone off the deep end a bit with this one, but I'm thinking I might have a winner.
People have been begging for a hunter bot for a while now, and while I could make one, I don't think I could do it justice. EQ2 is a very complex beast, with combat being especially complex. Despite what many think, it's *not* as simple as walking around, targeting stuff, and mashing buttons until you're done. When you play EQ2, and you fight things, you create little strategies out of the abilities you're given, and considering that there are 24 classes and umpteen billion ways of customizing (AA, etc), there are far too many of these strategies to bake in to the software.
So, in order to appeal to everyone, I'm going to stop working on the hunter bot, and start working on my new project, a group assistant (or you could call it a two-box helper).
This will, however, be like no ordinary two-box helper. My system will be completely customizable, down to every last detail, thanks to a custom rules engine I plan to build. I'm scrapping my DSL idea for this - it works, but I don't want to trust it for a production build, not yet - and going with an XML rules system with a visual designer on top of it.
This will not be a general purpose scripting application. The rules engine will support the project's only goal - to make two-boxing (or three, four, five, twenty, etc) a hands-off endeavor.
But I have ulterior motives as well... This application will facilitate a rules-based system upon which the final hunter-bot can be built. Once the rules engine is in place, and all the wiring is complete, it'll be go time.
More details to come later. =)
People have been begging for a hunter bot for a while now, and while I could make one, I don't think I could do it justice. EQ2 is a very complex beast, with combat being especially complex. Despite what many think, it's *not* as simple as walking around, targeting stuff, and mashing buttons until you're done. When you play EQ2, and you fight things, you create little strategies out of the abilities you're given, and considering that there are 24 classes and umpteen billion ways of customizing (AA, etc), there are far too many of these strategies to bake in to the software.
So, in order to appeal to everyone, I'm going to stop working on the hunter bot, and start working on my new project, a group assistant (or you could call it a two-box helper).
This will, however, be like no ordinary two-box helper. My system will be completely customizable, down to every last detail, thanks to a custom rules engine I plan to build. I'm scrapping my DSL idea for this - it works, but I don't want to trust it for a production build, not yet - and going with an XML rules system with a visual designer on top of it.
This will not be a general purpose scripting application. The rules engine will support the project's only goal - to make two-boxing (or three, four, five, twenty, etc) a hands-off endeavor.
But I have ulterior motives as well... This application will facilitate a rules-based system upon which the final hunter-bot can be built. Once the rules engine is in place, and all the wiring is complete, it'll be go time.
More details to come later. =)
Saturday, July 4, 2009
Moving!
Despite a number of promising leads, all of my job resources have run dry. As such, in three days, I am moving back home to Charleston, SC. My spirits are up, actually - I expected to feel thoroughly defeated (which I did), but at the moment I'm happy. I'll get to see my friends and family again. That's the important thing.
Meanwhile, I haven't let the out-of-work situation hold me back from programming. If anything, I'm more prolific on my own projects than I was able to be before, what with the 8-hr a day grind. Most recently, I'm working on a new EQ2-related project. The project itself is, well, top-secret at the moment - but I can discuss one of its major components: a script lexer and parser.
Compiler design and implementation is definitely one of my favorite topics in computing. I may be one of the few people in the world who enjoys hand-writing parsers. I can easily say that the grammar I wrote for this particular lexer/parser is the most complex grammar I've written with the intention of actually parsing.
In the span of about 3 hours tonight, I've hand-written the lexer and recursive descent parser for this language, which builds the inputs into a state machine. I can't give away too many details, but suffice it to say that this system will automate some interesting tasks in EQ2 (much like other MacroCrafter software I've written) and will allow the user to change its behavior on the fly.
To be honest, I don't really expect the user to do much editing - although I intend for the scripts to be user-editable, I will likely be writing all the scripts myself and releasing them with the new software. What I really wanted out of this was an easy way for me to describe the logic that this software will execute.
Sorry I'm being vague. I haven't made any official posts regarding this stuff yet on the MC forums, and I don't want to jump the gun. I've made the mistake in the past regarding talking about projects and such on this and other venues, getting people's hopes up, and then either abandoning or shelving said project. People's feelings get hurt, I get complaints, etc. I shouldn't promise what I might not deliver - and this vagueness is part of that.
As for when I'll be announcing this -- well, it'll come once I flesh the concept out more, build more proof of concept bits, and start pinning stuff together. I think it'll be well-received - possibly more so than a hunter bot would...
Meanwhile, I haven't let the out-of-work situation hold me back from programming. If anything, I'm more prolific on my own projects than I was able to be before, what with the 8-hr a day grind. Most recently, I'm working on a new EQ2-related project. The project itself is, well, top-secret at the moment - but I can discuss one of its major components: a script lexer and parser.
Compiler design and implementation is definitely one of my favorite topics in computing. I may be one of the few people in the world who enjoys hand-writing parsers. I can easily say that the grammar I wrote for this particular lexer/parser is the most complex grammar I've written with the intention of actually parsing.
In the span of about 3 hours tonight, I've hand-written the lexer and recursive descent parser for this language, which builds the inputs into a state machine. I can't give away too many details, but suffice it to say that this system will automate some interesting tasks in EQ2 (much like other MacroCrafter software I've written) and will allow the user to change its behavior on the fly.
To be honest, I don't really expect the user to do much editing - although I intend for the scripts to be user-editable, I will likely be writing all the scripts myself and releasing them with the new software. What I really wanted out of this was an easy way for me to describe the logic that this software will execute.
Sorry I'm being vague. I haven't made any official posts regarding this stuff yet on the MC forums, and I don't want to jump the gun. I've made the mistake in the past regarding talking about projects and such on this and other venues, getting people's hopes up, and then either abandoning or shelving said project. People's feelings get hurt, I get complaints, etc. I shouldn't promise what I might not deliver - and this vagueness is part of that.
As for when I'll be announcing this -- well, it'll come once I flesh the concept out more, build more proof of concept bits, and start pinning stuff together. I think it'll be well-received - possibly more so than a hunter bot would...
Friday, June 26, 2009
Out of work
Not sure why it took me so long to think about posting my current employment situation on my blog..... But here goes.
I am currently unemployed. Things didn't work out with my last company, and they let me go. I'm living in the Seattle, WA area, for at least the next few weeks, while I continue to look for work. If I can't find anything between now and then, we'll be packing up the family and moving back home to SC, unless there are better (read, nearly guaranteed) jobs elsewhere, in which case we'll probably relocate there instead.
If anyone who reads my blog is on a dev team looking for someone, and you think (by my posts) that I might be a good fit, please let me know! I would be forever appreciative. =)
I am currently unemployed. Things didn't work out with my last company, and they let me go. I'm living in the Seattle, WA area, for at least the next few weeks, while I continue to look for work. If I can't find anything between now and then, we'll be packing up the family and moving back home to SC, unless there are better (read, nearly guaranteed) jobs elsewhere, in which case we'll probably relocate there instead.
If anyone who reads my blog is on a dev team looking for someone, and you think (by my posts) that I might be a good fit, please let me know! I would be forever appreciative. =)
Monday, June 15, 2009
Retrospect
While working on converting my name generator from C# to F#, I've come across some strange things.
I mentioned earlier that the statistical analysis portion of my generator counts the number of syllables in its input set in order to use that data to make certain syllables appear more often than others. While looking through my C# code, it appears that this is not the case - I, in fact, am not using that data for anything.
It was my original intention to use it as such, but in retrospect I recall having issues translating this requirement into functionality - especially since I was using SQL Server to perform my random picking. I wonder if I can fulfill this requirement now that I'm using F# and eschewing SQL.
By the way - when I'm done with this project, I'll post up snippets of my F# for review... I'm sure you vets will have plenty to chuckle at! :)
I mentioned earlier that the statistical analysis portion of my generator counts the number of syllables in its input set in order to use that data to make certain syllables appear more often than others. While looking through my C# code, it appears that this is not the case - I, in fact, am not using that data for anything.
It was my original intention to use it as such, but in retrospect I recall having issues translating this requirement into functionality - especially since I was using SQL Server to perform my random picking. I wonder if I can fulfill this requirement now that I'm using F# and eschewing SQL.
By the way - when I'm done with this project, I'll post up snippets of my F# for review... I'm sure you vets will have plenty to chuckle at! :)
Saturday, June 13, 2009
F# progress
Made some progress on my name generator. The statistical analysis side is the hardest part - parsing individual words into their constituent syllables. I am happy to report that after several hours of plugging away, I have a mostly F#-axiomatic syllable parser based on the logic of my C# version. I made gratuitous use of discriminated unions, pattern matching, and recursive functions to write the word parsing logic, rather than OOP and procedural methods similar to what I used in C#. I feel like this has given me a decent grasp on what the language is capable of.
My next task is to continue the statistical analysis portion, and generate syllable boundary data (which is important for the generator to know how to put words together in ways that make sense). After that, the generator itself needs to be written, which should be a simple matter.
The only question at this point that I have is how to handle the SQL database access - much of the logic for generation is embedded in the SQL calls to the database. I'm considering eschewing the database completely and designing my own file-based storage solution, but then I have a whole new class of problems.
In the end, this is going to get compiled as a class library, for consumption in my name generator website (which currently isn't online).
My next task is to continue the statistical analysis portion, and generate syllable boundary data (which is important for the generator to know how to put words together in ways that make sense). After that, the generator itself needs to be written, which should be a simple matter.
The only question at this point that I have is how to handle the SQL database access - much of the logic for generation is embedded in the SQL calls to the database. I'm considering eschewing the database completely and designing my own file-based storage solution, but then I have a whole new class of problems.
In the end, this is going to get compiled as a class library, for consumption in my name generator website (which currently isn't online).
New F# project
Despite the fact that work has intervened in almost all my personal project plans (mostly in the form of making me want to do anything but programming while I'm at home), I still want to learn a new language. F# has been on my radar for a while, especially since it's being included with vs2010. I have been struggling with trying to find something to write in F#, something that I know well enough to be useful as a tutorial for myself, but also something large enough that I'll be able to use most of the idioms in F# that I should need for most any large project.
I had forgotten that, apart from simple expression parsers, I used my Random Name Generator project to learn new languages. My name generator requires text input in order to build statistical data from which random words are generated. It was originally written in Visual Basic 6.0, and was ported to VB.NET, then to C#, in order for me to learn those respective languages. I think it's time to do it again. =)
I had forgotten that, apart from simple expression parsers, I used my Random Name Generator project to learn new languages. My name generator requires text input in order to build statistical data from which random words are generated. It was originally written in Visual Basic 6.0, and was ported to VB.NET, then to C#, in order for me to learn those respective languages. I think it's time to do it again. =)
Thursday, June 11, 2009
I hate release days...
My months-long shipping project has just been released - my largest at this company to date - and it replaced a great deal of critical functionality. My nerves are shot and I didn't sleep well last night while our lead dev rolled out the changes. All went well, and everything's fine, mostly thanks to the fact that we 'practiced' the release the day before on our test setup.
Still, the butterflies won't go away...
Still, the butterflies won't go away...
Friday, May 22, 2009
.NET 4 + VS 2010 = Sweeeeet
-- and it's that sweetness that makes me feel even more guilty about having a VS niggle after 2 minutes of using it...
In VS 2008, if, in the body of a method, you press Enter a few times to give yourself some space, when you up-arrow to go back up, the cursor stays at the indentation level of the rest of the method. 2010 doesn't do this - the cursor jumps to position 1, regardless of the current indentation level you're at.
Meh.
In VS 2008, if, in the body of a method, you press Enter a few times to give yourself some space, when you up-arrow to go back up, the cursor stays at the indentation level of the rest of the method. 2010 doesn't do this - the cursor jumps to position 1, regardless of the current indentation level you're at.
Meh.
Monday, May 18, 2009
Behavior Driven Design
Every now and then I get the urge to try out some of the more recent design methods coming out lately - usually they tend to end in 'DD.' Rob Conery is largely responsible for this trend in me to experiment, as his screencast series on MVC Storefront (now Kona) demo these different ideas rather well.
His most recent screencast introduced Behavior Driven Design (BDD) to me, and I have to say that of all the 'DD's, BDD makes the most sense to me. So I've decided to try and build an entire project using it (one that I've been putting off far too long).
The guild to which I belong (which I will not name here) requires a website, and rather than use something like GuildPortal for it, I've decided to build it myself, using ASP.NET MVC. All of the various systems will be custom built by me - I don't want to use any community management software, etc., because I want this to be a learning experience for me.
Among the components I'll be building, the simplest is the News Service, which will let administrators post guild news. I've decided to spec out this service before I build it. Here's my list of specs:
This describes a basic news service, I think, as a user or administrator might understand it (which is one of the goals of BDD btw - the terminology you use should be part of the business language rather than the technical language).
Now for some disclosure - I must admit I've cheated a bit here for the purposes of this blog post. The above list comes from the MSpec runner report output. MSpec is a BDD specification framework which lets you define your specifications in terms of executable code, which you can use to verify that your specifications are met. I did write my specs first - MSpec allows you to leave your specifications unimplemented - but I have already written the required functionality against a mock news article repository.
So far - fun stuff. I'm still a little apprehensive - I have no real idea what I'm doing, but I'm sure I'll figure it out as I make false steps here and there. This is a toy project really (but unlike other toy projects this one will be actually useful, so I'll be less inclined to give up on it, especially with my guild leader nagging me incessantly...) - but I want to give it all I can.
His most recent screencast introduced Behavior Driven Design (BDD) to me, and I have to say that of all the 'DD's, BDD makes the most sense to me. So I've decided to try and build an entire project using it (one that I've been putting off far too long).
The guild to which I belong (which I will not name here) requires a website, and rather than use something like GuildPortal for it, I've decided to build it myself, using ASP.NET MVC. All of the various systems will be custom built by me - I don't want to use any community management software, etc., because I want this to be a learning experience for me.
Among the components I'll be building, the simplest is the News Service, which will let administrators post guild news. I've decided to spec out this service before I build it. Here's my list of specs:
- when viewing article titles
- should display a date descending ordered list of article titles
- when viewing top n articles
- should display up to n articles in a date descending ordered list
- when creating a new article
- should contain the authors id
- should contain the posted date
- should contain the article title
- should contain the article body
- when viewing a single article
- should display the articles contents
- when adding a new news article
- should provide a new id for the news article
- should add the item to the news service
- when removing a news article
- should remove the item from the news service
- when editing a news article
- should update the item to the new contents
This describes a basic news service, I think, as a user or administrator might understand it (which is one of the goals of BDD btw - the terminology you use should be part of the business language rather than the technical language).
Now for some disclosure - I must admit I've cheated a bit here for the purposes of this blog post. The above list comes from the MSpec runner report output. MSpec is a BDD specification framework which lets you define your specifications in terms of executable code, which you can use to verify that your specifications are met. I did write my specs first - MSpec allows you to leave your specifications unimplemented - but I have already written the required functionality against a mock news article repository.
So far - fun stuff. I'm still a little apprehensive - I have no real idea what I'm doing, but I'm sure I'll figure it out as I make false steps here and there. This is a toy project really (but unlike other toy projects this one will be actually useful, so I'll be less inclined to give up on it, especially with my guild leader nagging me incessantly...) - but I want to give it all I can.
File operations and temp folders...
This is a problem I've seen before in many places, and it seems like good practice (or at least it must have at some point) - but for me it seems to cause more harm than good.
Many program (Internet Explorer for file downloads, certain ZIP utilities, etc) perform file operations in a temporary folder, then copy the results of the operation to the ultimate destination.
I can see why this makes sense. Depending on the operation, allowing the user to muck with the file during processing can cause data loss, errors, mini-black holes, etc. But when the resulting file is very, very large, the subsequent copy can take forever. Additionally, if there isn't enough space on the destination, then instead of failing fast and reporting the file allocation error, we have to wait until the end of a potentially expensive operation (like unzipping a 12 gig file from an archive) before we find out. In fact, this practice makes such a problem more likely because the space requirements are doubled in order to facilitate the copy.
I'm really just ranting here rather than offering a fix for the problem - the only fix I could possibly suggest would be to avoid doing this at all. If you must, however, make it an option for the user to override, allowing said user to take responsibility for his/her own file system.
*ahem* STOP PROTECTING ME FROM MYSELF.
Many program (Internet Explorer for file downloads, certain ZIP utilities, etc) perform file operations in a temporary folder, then copy the results of the operation to the ultimate destination.
I can see why this makes sense. Depending on the operation, allowing the user to muck with the file during processing can cause data loss, errors, mini-black holes, etc. But when the resulting file is very, very large, the subsequent copy can take forever. Additionally, if there isn't enough space on the destination, then instead of failing fast and reporting the file allocation error, we have to wait until the end of a potentially expensive operation (like unzipping a 12 gig file from an archive) before we find out. In fact, this practice makes such a problem more likely because the space requirements are doubled in order to facilitate the copy.
I'm really just ranting here rather than offering a fix for the problem - the only fix I could possibly suggest would be to avoid doing this at all. If you must, however, make it an option for the user to override, allowing said user to take responsibility for his/her own file system.
*ahem* STOP PROTECTING ME FROM MYSELF.
Sunday, May 3, 2009
Genetic Algorithm + Brainfsck
First: Genetic Algorithms. A GA is a program designed to find the solution to a problem via iterative searching - that is, the algorithm itself searches for a solution to a problem. GAs in particular use biologically-inspired methods to perform the search.
The analogy with biological systems is like so: a GA spawns multiple search 'organisms,' which 'live' in the algorithmic environment. With each generation, each organism is tested against a 'fitness' function, which determines how well the organism solves the problem we're looking for a solution for. Only the fittest of each generation survive, passing on their genes (parameters) to the next generation. Each of the survivors randomly selects a mate and mixes the genes (with some random mutation factors), with the expected outcome of iteratively evolving toward the best possible solution to the problem.
I've built a couple small GA systems in the past - nothing for anything in production, but mostly just to toy with - and I find them particularly interesting, especially given how very small changes in how the organisms select mates, mix genes, and mutate can result in large-scale changes in the search behavior.
Now the second part: Brainfsck (which is actually spelled with a 'u' instead of an 's', but I want to maintain some civility here) is a programming language in which there are only 8 legal characters - '>' '<' '+' '-' '.' ',' '[' and ']'. Each of these characters performs some function in the program, usually with regards to manipulating 8-bit memory locations called cells. Wikipedia has more information - but suffice it to say that Brainfsck has been proven to be Turing complete, which means it can be used to perform any computable function (provided it has infinite memory).
Here's how Genetic Algorithms and Brainfsck can come together: I mentioned earlier that GAs use digital organisms and 'genetic' parameters to evolve toward a solution. What if those parameters were snippets of Brainfsck programs, and the fitness function ran the Brainfsck programs to see what their output is, and check that output against an expected output? Wouldn't it, in theory, be possible to iteratively evolve toward the solution of any computation problem that way?
Just a random thought that's been plaguing me for the past couple of days. Don't think anything particularly interesting would come of this.
The analogy with biological systems is like so: a GA spawns multiple search 'organisms,' which 'live' in the algorithmic environment. With each generation, each organism is tested against a 'fitness' function, which determines how well the organism solves the problem we're looking for a solution for. Only the fittest of each generation survive, passing on their genes (parameters) to the next generation. Each of the survivors randomly selects a mate and mixes the genes (with some random mutation factors), with the expected outcome of iteratively evolving toward the best possible solution to the problem.
I've built a couple small GA systems in the past - nothing for anything in production, but mostly just to toy with - and I find them particularly interesting, especially given how very small changes in how the organisms select mates, mix genes, and mutate can result in large-scale changes in the search behavior.
Now the second part: Brainfsck (which is actually spelled with a 'u' instead of an 's', but I want to maintain some civility here) is a programming language in which there are only 8 legal characters - '>' '<' '+' '-' '.' ',' '[' and ']'. Each of these characters performs some function in the program, usually with regards to manipulating 8-bit memory locations called cells. Wikipedia has more information - but suffice it to say that Brainfsck has been proven to be Turing complete, which means it can be used to perform any computable function (provided it has infinite memory).
Here's how Genetic Algorithms and Brainfsck can come together: I mentioned earlier that GAs use digital organisms and 'genetic' parameters to evolve toward a solution. What if those parameters were snippets of Brainfsck programs, and the fitness function ran the Brainfsck programs to see what their output is, and check that output against an expected output? Wouldn't it, in theory, be possible to iteratively evolve toward the solution of any computation problem that way?
Just a random thought that's been plaguing me for the past couple of days. Don't think anything particularly interesting would come of this.
Friday, March 27, 2009
DLR Trees == LINQ Expression Trees?
Appears so - at least, once .NET 4 ships.
I've been doing some research on the DLR lately, for no other reason than sheer boredom - and came away fascinated. The DLR (Dynamic Language Runtime) is .NET's answer to dynamic languages running in the .NET environment. It's still in beta at this time (DLR is version .9 as of this writing), but already many languages have been implemented on it - the most famous of which are the Iron languages - IronPython and IronRuby.
The way it works (and I'm simplifying things here) is thus: you provide the translation from source code to DLR trees, and the DLR provides IL generation, fast execution, a proven garbage collector, and the entire .NET Framework. Sounds like a bargain to me. DLR trees are syntax trees that tell the DLR how to generate code - you have things like AssignmentExpression, StatementExpression, LambdaExpression, etc.
A lot of this sounds very similar to what you get right now with LINQ Expression trees. In fact, there was an announcement a short time ago which stated that LINQ Expression trees were going to be getting an upgrade. Currently (.NET 3.5) LINQ Expression trees can only represent simple logic - property lookups, method invocations, things like that. You currently cannot express an IF statement, for example, inside a LINQ Expression tree. With .NET 4.0, that's going to be changing - LINQ Expression trees will be augmented with the functionality it's currently missing.
This is great news for LINQ implementors, to be sure - but seems to me to be a bit redundant. LINQ expression trees and DLR trees are starting to sound a lot alike - so I popped on over to the DLR's home on CodePlex and asked about it:
Me:
Them (specifically, Bill Chiles):
So, there you have it - DLR Trees == LINQ Expression Trees. Very cool stuff happens when you explore the possibilities here - consider a LINQ to SQL provider that might be capable of understanding your custom C# logic and translating that into the equivalent TSQL code -- or, even better -- send the DLR tree over the wire to SQL Server which can run it on a DLR implementation there!
That would be just awesome...
Here is the thread on CodePlex where I chat with Bill about the trees, in case you'd like to see the follow-ups.
I've been doing some research on the DLR lately, for no other reason than sheer boredom - and came away fascinated. The DLR (Dynamic Language Runtime) is .NET's answer to dynamic languages running in the .NET environment. It's still in beta at this time (DLR is version .9 as of this writing), but already many languages have been implemented on it - the most famous of which are the Iron languages - IronPython and IronRuby.
The way it works (and I'm simplifying things here) is thus: you provide the translation from source code to DLR trees, and the DLR provides IL generation, fast execution, a proven garbage collector, and the entire .NET Framework. Sounds like a bargain to me. DLR trees are syntax trees that tell the DLR how to generate code - you have things like AssignmentExpression, StatementExpression, LambdaExpression, etc.
A lot of this sounds very similar to what you get right now with LINQ Expression trees. In fact, there was an announcement a short time ago which stated that LINQ Expression trees were going to be getting an upgrade. Currently (.NET 3.5) LINQ Expression trees can only represent simple logic - property lookups, method invocations, things like that. You currently cannot express an IF statement, for example, inside a LINQ Expression tree. With .NET 4.0, that's going to be changing - LINQ Expression trees will be augmented with the functionality it's currently missing.
This is great news for LINQ implementors, to be sure - but seems to me to be a bit redundant. LINQ expression trees and DLR trees are starting to sound a lot alike - so I popped on over to the DLR's home on CodePlex and asked about it:
Me:
Not sure if this is the right place to ask this, but here goes:
Why is there so much duplication between these two incredibly similar tasks? Why couldn't Expression Trees have been implemented as DLR Trees (or vice versa)?
Them (specifically, Bill Chiles):
:-) You’re like the audience plant to ask just the right questions :-).
They are the same. LINQ Expr Trees v1 evolved into Expr Trees v2 which are exactly the DLR tress. All the sources you see in our codeplex project are the sources we’re shipping in CLR 4.0 for all of the Expr Trees code. What might be confusing is that until we RTM CLR 4.0, we change the namespaces on codeplex. We need to do that so that if you have a .NET 3.5 C# app that both uses LINQ and hosts, say, IronPython, then LINQ works as well as the DLR. You just can hand those threes back and forth, which would be a very corner case scenario if one at all. When we hit RTM, the codeplex sources will will have the same namepace, but we won’t build the Microsoft.scripting.core.dll in our .sln file.
Bill
So, there you have it - DLR Trees == LINQ Expression Trees. Very cool stuff happens when you explore the possibilities here - consider a LINQ to SQL provider that might be capable of understanding your custom C# logic and translating that into the equivalent TSQL code -- or, even better -- send the DLR tree over the wire to SQL Server which can run it on a DLR implementation there!
That would be just awesome...
Here is the thread on CodePlex where I chat with Bill about the trees, in case you'd like to see the follow-ups.
Wednesday, March 18, 2009
Tuesday, March 3, 2009
Long silence
It's been over a month since my last post - I'm sorry about this. I have a horrible tendency to forget my blogs, as my MySpace friends can attest to. I have an excuse this time though - my workload has effectively doubled at the office, and I've had to budget my time recently.
I also spent some time down at my company's warehouse getting to know the warehouse staff and seeing the physical processes that drive the company. It was definitely an eye-opening experience - this was the first time I've ever visited a warehouse of any sort, and seeing them physically carry out the processes that I've helped to develop was gratifying in a way. It helped me to fully understand the ramifications of the decisions that I make when building the software that they'll use.
My spare time (hah!) has been spent getting the new bot up and running - so far, so good! :) I have a command-line version up and running that is fully configurable and about 70% functional - no hunting functionality is in, but harvesting works a charm. I've opened up beta applications on the Macrocrafter website, so you should hop over there if you're interested in helping me test.
The development process has been much easier this time around, I think. I wasn't very smart with the first few iterations of the harvest bot - during the alpha and beta periods I pretty much wrote EverHarvest by the seat of my pants, largely without a plan at all. The result was that after the command-line tests were complete, I had to completely rewrite the system because I had coupled it into the command-line project. This time, I'm developing the engine of the system completely separate from the UI - it's in its own separate assembly - which will make it *much* easier to move from the command-line interface to the graphical interface.
So these are the developments of late. Nothing earth-shattering, but lots of things to distract me from blogging. I'll try to keep up moving forward. =)
I also spent some time down at my company's warehouse getting to know the warehouse staff and seeing the physical processes that drive the company. It was definitely an eye-opening experience - this was the first time I've ever visited a warehouse of any sort, and seeing them physically carry out the processes that I've helped to develop was gratifying in a way. It helped me to fully understand the ramifications of the decisions that I make when building the software that they'll use.
My spare time (hah!) has been spent getting the new bot up and running - so far, so good! :) I have a command-line version up and running that is fully configurable and about 70% functional - no hunting functionality is in, but harvesting works a charm. I've opened up beta applications on the Macrocrafter website, so you should hop over there if you're interested in helping me test.
The development process has been much easier this time around, I think. I wasn't very smart with the first few iterations of the harvest bot - during the alpha and beta periods I pretty much wrote EverHarvest by the seat of my pants, largely without a plan at all. The result was that after the command-line tests were complete, I had to completely rewrite the system because I had coupled it into the command-line project. This time, I'm developing the engine of the system completely separate from the UI - it's in its own separate assembly - which will make it *much* easier to move from the command-line interface to the graphical interface.
So these are the developments of late. Nothing earth-shattering, but lots of things to distract me from blogging. I'll try to keep up moving forward. =)
Friday, January 30, 2009
I keep learning new things about C#
Apparently, in the body of a method, you can arbitrarily use braces to create 'pockets' of child-local scope. For example:
This will, as expected, create the output 11 and 12. The variable 'y' is created twice, but in each case it's created in some sort of anonymous child scope. I was always under the impression that you had to have a reason of some sort in order to create child scope like this. Glad to see I was mistaken - this might come in handy some day...
static void Main(string[] args) {
int x = 10;
{
int y = 1;
Console.WriteLine(x + y);
}
{
int y = 2;
Console.WriteLine(x + y);
}
Console.ReadLine();
}
This will, as expected, create the output 11 and 12. The variable 'y' is created twice, but in each case it's created in some sort of anonymous child scope. I was always under the impression that you had to have a reason of some sort in order to create child scope like this. Glad to see I was mistaken - this might come in handy some day...
Wednesday, January 21, 2009
Progress
To cut to the chase - progress on the new harvest/hunter bot is going very slow. Obviously I can't work on it while at work - sort of a conflict of interest when I'm supposed to be working on work stuff while at work - so my availability is limited. Weekends are limited as well, since because I work a 'real job' I only get to really see the kids during the weekends, and because the last thing I want to do during my days off is work...
More technically, I'm blocked on a couple issues. Since I'm merging a harvest bot and a hunter bot together, there are some technical difficulties to work out - they have to share targeting systems, navigation systems, and some settings which, from a UI perspective, will have to be separated yet united somehow... As far as the engine itself goes - well, because I'm using my workflow system, it's actually not going to be as hard as it seems.
I've been using the workflows as logical separations between the various systems involved. One workflow for hunter, one for harvester, and one for navigation. Targeting is too simple a process for an entire workflow - a single workitem will be sufficient. Finally, to tie everything together, a master workflow responsible for coordinating the efforts of these disparate (yet united) tasks.
The master workflow will be responsible for invoking the navigation sub-workflow, as well as the harvester and hunter workflows. All workitems for these sub-workflows are yielded to the workflow runner, so the entire hierarchy of workflows is flattened to a sequential list as its run - as it should be.
Where I'm being held up is in how to handle tasks that should be shared by all these workflows, but must happen separately in each of them. For example, stuck checking / handling. This sounds like a navigation concern, and it is, but it's also a concern for harvesting and hunting - in particular, moving from the walkpath to a node / mob, and moving back to the walkpath. Both of those tasks are prone to dealing with the stuck issue, so I'll need to have stuck checking there as well. Should I create another workflow - a stuck check workflow - that's responsible for detecting and dealing with the issue? If so, how do I invoke it while ensuring the harvest, hunter, and navigation workflows are ignorant of it? That's one of the goals for workflows - keeping the sub-workflows ignorant to each other and letting the master workflow do the coordination.
Maybe that is the answer. I can inspect the workitems given to me from the Hunter and Harvest workflows before I yield them - maybe I can check the specifics, and if they're in their navigation phases I can interleave the stuck checking in there with them. Hah - yes, I'll do it. =) Sometimes just talking through a problem can lead you to a solution - I'll give it a try when I get home.
Well, anyway - I need to get back to work. Lunch break only lasts so long...
More technically, I'm blocked on a couple issues. Since I'm merging a harvest bot and a hunter bot together, there are some technical difficulties to work out - they have to share targeting systems, navigation systems, and some settings which, from a UI perspective, will have to be separated yet united somehow... As far as the engine itself goes - well, because I'm using my workflow system, it's actually not going to be as hard as it seems.
I've been using the workflows as logical separations between the various systems involved. One workflow for hunter, one for harvester, and one for navigation. Targeting is too simple a process for an entire workflow - a single workitem will be sufficient. Finally, to tie everything together, a master workflow responsible for coordinating the efforts of these disparate (yet united) tasks.
The master workflow will be responsible for invoking the navigation sub-workflow, as well as the harvester and hunter workflows. All workitems for these sub-workflows are yielded to the workflow runner, so the entire hierarchy of workflows is flattened to a sequential list as its run - as it should be.
Where I'm being held up is in how to handle tasks that should be shared by all these workflows, but must happen separately in each of them. For example, stuck checking / handling. This sounds like a navigation concern, and it is, but it's also a concern for harvesting and hunting - in particular, moving from the walkpath to a node / mob, and moving back to the walkpath. Both of those tasks are prone to dealing with the stuck issue, so I'll need to have stuck checking there as well. Should I create another workflow - a stuck check workflow - that's responsible for detecting and dealing with the issue? If so, how do I invoke it while ensuring the harvest, hunter, and navigation workflows are ignorant of it? That's one of the goals for workflows - keeping the sub-workflows ignorant to each other and letting the master workflow do the coordination.
Maybe that is the answer. I can inspect the workitems given to me from the Hunter and Harvest workflows before I yield them - maybe I can check the specifics, and if they're in their navigation phases I can interleave the stuck checking in there with them. Hah - yes, I'll do it. =) Sometimes just talking through a problem can lead you to a solution - I'll give it a try when I get home.
Well, anyway - I need to get back to work. Lunch break only lasts so long...
Thursday, January 15, 2009
Little accomplishments
It's great when I get to put new things I've learned to work, especially when it's to the greater good.
Case in point - did you know that assignment operations return the result of the assignment? I didn't until recently - it's one of those obscure language bits that almost never come up, but when they do they can really make life easier.
I'm working on my workflow idea (see my previous post for details) - in particular, I'm toying with the idea of composing workflows from sub-workflows - and so I wrote a workflow that takes two workflows and interleaves their workitems together. At first I used a 'Zip' method which does the job, but if there are more workitems in one workflow than the other, the extras are truncated - this is not what I wanted; I want the extra workitems returned correctly.
So I wrote this Workflow() method, which does the job:
Interleaving workflow that yields extras
It works, but is verbose and ugly. It is very much imperative code - the *how* of the code obscures much of the *what* and the *why*.
Obscure language features come to the rescue, however, when a little light-bulb goes off in my head. Because assignment operators return the result of the assignment, I can combine the assignment and the test in one statement. The original code becomes this:
From 14 LOC to 6 - good stuff
This may seem like child's play to some of you, and you're probably right, but for me, this represents an accomplishment (albeit small) in my understanding of the language I use.
Case in point - did you know that assignment operations return the result of the assignment? I didn't until recently - it's one of those obscure language bits that almost never come up, but when they do they can really make life easier.
I'm working on my workflow idea (see my previous post for details) - in particular, I'm toying with the idea of composing workflows from sub-workflows - and so I wrote a workflow that takes two workflows and interleaves their workitems together. At first I used a 'Zip' method which does the job, but if there are more workitems in one workflow than the other, the extras are truncated - this is not what I wanted; I want the extra workitems returned correctly.
So I wrote this Workflow() method, which does the job:
do {
if (enum1.MoveNext()) {
yield return enum1.Current;
} else {
enum1Done = true;
}
if (enum2.MoveNext()) {
yield return enum2.Current;
} else {
enum2Done = true;
}
} while (!(enum1Done && enum2Done));
It works, but is verbose and ugly. It is very much imperative code - the *how* of the code obscures much of the *what* and the *why*.
Obscure language features come to the rescue, however, when a little light-bulb goes off in my head. Because assignment operators return the result of the assignment, I can combine the assignment and the test in one statement. The original code becomes this:
var enum1Alive = false;
var enum2Alive = false;
while ((enum1Alive = enum1.MoveNext()) | (enum2Alive = enum2.MoveNext())) {
if (enum1Alive) yield return enum1.Current;
if (enum2Alive) yield return enum2.Current;
}
This may seem like child's play to some of you, and you're probably right, but for me, this represents an accomplishment (albeit small) in my understanding of the language I use.
Wednesday, January 7, 2009
Yield, and the C# state machine
For those of you who don't already know, the C# compiler has its own state machine generator that you can use. It's true - it's called the 'yield' statement!
Yield is used to create implementations of the Enumerable pattern - a software pattern that allows you to treat a collection of things as an enumeration, over which you can perform some process. In C#, you consume an enumeration via the 'foreach' statement, like so:
Iterating over a List of strings
Before C# 2.0, creating a custom enumerable meant implementing the Enumerable Pattern via IEnumerator and IEnumerable. I don't feel like going through this, so here's a short and sweet example I found online.
This is a fairly common implementation. In fact, it is so common that when the C# language designers were conceiving 2.0 of their product, they chose to make it a first-class compiler-driven feature. Enter the yield keyword, which is capable of turning the example above into the following code:
Implementing IEnumerable and IEnumerator via the yield keyword
This is much more straightforward, but there is some magic happening here that allows this to happen. First of all, the Enumerable pattern includes the requirement that the processing of the enumeration be lazy - that is, evaluated on a need basis. This allows an enumeration to contain an effectively infinite amount of items. Such an enumeration can be created using this code:
Creating an enumeration of an infinite pattern of alternating booleans
This code generates a list of alternating booleans - True / False / True / False - forever. Surely, this code will result in a locked-up process. Not so, fortunately for us, because behind the scenes (in the magic part) this code is expanded into a proper implementation of the Enumerable pattern - lazy evaluation included. Only when you request the next value is it generated and then provided, meaning this infinite generation can be short-circuited at any moment.
How? Magic, like I said - although this magic can be explained through gratuitous use of .NET Reflector. According to .NET Reflector, my infiniteAlternatingBools() method looks like this:
Reflector output of infiniteAlternatingBools method
What? This is a mess. What is reflector telling me about this code?
In a nutshell, Reflector is saying that the C# compiler has, behind my back, taken the code I wrote and moved it into an anonymous private class. The constructor of that class takes an integer in its constructor, which the rewritten method initializes to -2. It also sets a 'this' property to the class that contains infiniteAlternatingBools - probably allowing it to access the original class's private members. Then the rewritten method returns the instance of that anonymous class - which suggests that it implements IEnumerable<bool>.
Kind of rude, don't you think? Replacing our carefully written infinite loop with some object creation? Actually the C# compiler has done us a favor - if you look in the anonymous class it generated you'll find the original code you wrote, albeit in a form you might not fully recognize. Here's the listing of the class (cleaned up a little bit from the Reflector version, which contains illegal characters):
Listing of the generated anonymous class implementing the Enumerable pattern
This listing is a bit hard to understand. There are fields called state, current, _this, initialThreadID, and _6. There are the IEnumerable and IEnumerator implementations - MoveNext, Current, GetEnumerator, and Reset. There's a constructor (taking an int). What can all this mean? More importantly, where's my infinite loop?
There is no infinite loop. My code is still here, but it's been turned into a state machine. The value that gets passed in to the constructor (-2) tells this state machine that it's in the initial state. When GetEnumerator is called, it checks to see if it's in its initial state - if it's not, it creates a new version of itself and returns that - but if it is, then it moves into state '0' and returns itself. When MoveNext is called, it uses the state to determine what value to set as the 'current' property. At state 0, the initial value is returned - which the C# compiler correctly determined to be false, given my initial 'bool cur = false;' statement. It also moves into state 1. Subsequent calls to MoveNext will call my code which alternates this _6 value, which is a boolean, between true and false - mimicking the behavior I coded.
My infinite loop turned into a lazily evaluated Enumerable Pattern implementation which uses a state machine to decide on what the 'current' value should be whenever MoveNext is called. pretty damned cool if you ask me.
The coolest thing about this is that you can use any C# constructs you want in your enumerable, and create some incredibly complex generators. In my case, I'm taking advantage of the built-in state machine to create a workflow-like process. One place I plan on using this is in EverHarvest 2. Here's a simplified version of what my workflow might look like once fully implemented:
EverHarvest Workflow example
Notice how simple this is to understand. It reads very procedurally, and yet because this is lazily evaluated, this process can be interrupted at any of the yield points. This control lies with the code that is enumerating through the workflow - that code can act as the gatekeeper, deciding when to get the next work item, when to execute it, when to break out of the loop, what data each bit should have, etc. This can be done in a foreach statement, or I can use the older MoveNext / Current members.
I hope this helps those of you who are still reading to understand how the yield statement can be used to take advantage of the state machine functionality that the C# compiler provides for us. In terms of readability and maintenance, it has proven to be a real boon for me. I hope this has helped you to find the same benefit.
Edit: Here's the disassembled MoveNext() method from Reflector - I haven't cleaned it up a bit. Lots of red squiglies in this one...
Reflected MoveNext() method
Yield is used to create implementations of the Enumerable pattern - a software pattern that allows you to treat a collection of things as an enumeration, over which you can perform some process. In C#, you consume an enumeration via the 'foreach' statement, like so:
1 IEnumerable<string> ies = new List<string>() { "asd", "ert", "qwe", "fgh" };
2
3 foreach (var s in ies) {
4 Console.WriteLine(s);
5 }
Before C# 2.0, creating a custom enumerable meant implementing the Enumerable Pattern via IEnumerator and IEnumerable. I don't feel like going through this, so here's a short and sweet example I found online.
This is a fairly common implementation. In fact, it is so common that when the C# language designers were conceiving 2.0 of their product, they chose to make it a first-class compiler-driven feature. Enter the yield keyword, which is capable of turning the example above into the following code:
1 public IEnumerable<char> MyChars() {
2 yield return 'A';
3 yield return 'B';
4 yield return 'C';
5 yield return 'D';
6 }
This is much more straightforward, but there is some magic happening here that allows this to happen. First of all, the Enumerable pattern includes the requirement that the processing of the enumeration be lazy - that is, evaluated on a need basis. This allows an enumeration to contain an effectively infinite amount of items. Such an enumeration can be created using this code:
1 public IEnumerable<bool> infiniteAlternatingBools() {
2 bool cur = false;
3 while (true) {
4 yield return cur;
5 cur = !cur;
6 }
7 }
This code generates a list of alternating booleans - True / False / True / False - forever. Surely, this code will result in a locked-up process. Not so, fortunately for us, because behind the scenes (in the magic part) this code is expanded into a proper implementation of the Enumerable pattern - lazy evaluation included. Only when you request the next value is it generated and then provided, meaning this infinite generation can be short-circuited at any moment.
How? Magic, like I said - although this magic can be explained through gratuitous use of .NET Reflector. According to .NET Reflector, my infiniteAlternatingBools() method looks like this:
1 public IEnumerable<bool> infiniteAlternatingBools()
2 {
3 <infiniteAlternatingBools>d__5 d__ = new <infiniteAlternatingBools>d__5(-2);
4 d__.<>4__this = this;
5 return d__;
6 }
What? This is a mess. What is reflector telling me about this code?
In a nutshell, Reflector is saying that the C# compiler has, behind my back, taken the code I wrote and moved it into an anonymous private class. The constructor of that class takes an integer in its constructor, which the rewritten method initializes to -2. It also sets a 'this' property to the class that contains infiniteAlternatingBools - probably allowing it to access the original class's private members. Then the rewritten method returns the instance of that anonymous class - which suggests that it implements IEnumerable<bool>.
Kind of rude, don't you think? Replacing our carefully written infinite loop with some object creation? Actually the C# compiler has done us a favor - if you look in the anonymous class it generated you'll find the original code you wrote, albeit in a form you might not fully recognize. Here's the listing of the class (cleaned up a little bit from the Reflector version, which contains illegal characters):
1 [CompilerGenerated]
2 private sealed class d__5 :
3 IEnumerable<bool>, IEnumerable,
4 IEnumerator<bool>, IEnumerator,
5 IDisposable {
6
7 // Fields
8 private int state;
9 private bool current;
10 public Program.anon _this;
11 private int initialThreadId;
12 public bool _6;
13
14 // Methods
15 [DebuggerHidden]
16 public d__5(int state) {
17 this.state = state;
18 this.initialThreadId = Thread.CurrentThread.ManagedThreadId;
19 }
20
21 public bool MoveNext() {
22 switch (this.state) {
23 case 0:
24 this.state = -1;
25 this._6 = false;
26 break;
27
28 case 1:
29 this.state = -1;
30 this._6 = !this._6;
31 break;
32
33 default:
34 return false;
35 }
36 this.current = this._6;
37 this.state = 1;
38 return true;
39 }
40
41 [DebuggerHidden]
42 IEnumerator<bool> IEnumerable<bool>.GetEnumerator() {
43 if ((Thread.CurrentThread.ManagedThreadId == this.initialThreadId) && (this.state == -2)) {
44 this.state = 0;
45 return this;
46 }
47 Program.anon.d__5 d__ = new Program.anon.d__5(0);
48 d__._this = this._this;
49 return d__;
50 }
51
52 [DebuggerHidden]
53 IEnumerator IEnumerable.GetEnumerator() {
54 return this;
55 }
56
57 [DebuggerHidden]
58 void IEnumerator.Reset() {
59 throw new NotSupportedException();
60 }
61
62 void IDisposable.Dispose() {
63 }
64
65 // Properties
66 bool IEnumerator<bool>.Current {
67 [DebuggerHidden]
68 get {
69 return this.current;
70 }
71 }
72
73 object IEnumerator.Current {
74 [DebuggerHidden]
75 get {
76 return this.current;
77 }
78 }
79 }
This listing is a bit hard to understand. There are fields called state, current, _this, initialThreadID, and _6. There are the IEnumerable and IEnumerator implementations - MoveNext, Current, GetEnumerator, and Reset. There's a constructor (taking an int). What can all this mean? More importantly, where's my infinite loop?
There is no infinite loop. My code is still here, but it's been turned into a state machine. The value that gets passed in to the constructor (-2) tells this state machine that it's in the initial state. When GetEnumerator is called, it checks to see if it's in its initial state - if it's not, it creates a new version of itself and returns that - but if it is, then it moves into state '0' and returns itself. When MoveNext is called, it uses the state to determine what value to set as the 'current' property. At state 0, the initial value is returned - which the C# compiler correctly determined to be false, given my initial 'bool cur = false;' statement. It also moves into state 1. Subsequent calls to MoveNext will call my code which alternates this _6 value, which is a boolean, between true and false - mimicking the behavior I coded.
My infinite loop turned into a lazily evaluated Enumerable Pattern implementation which uses a state machine to decide on what the 'current' value should be whenever MoveNext is called. pretty damned cool if you ask me.
The coolest thing about this is that you can use any C# constructs you want in your enumerable, and create some incredibly complex generators. In my case, I'm taking advantage of the built-in state machine to create a workflow-like process. One place I plan on using this is in EverHarvest 2. Here's a simplified version of what my workflow might look like once fully implemented:
1 public IEnumerable<WorkUnit> Workflow() {
2 yield return new Initialize();
3
4 while (true) {
5 var wp = GetNextWaypoint();
6 var wtw = new WalkingToWaypoint(wp);
7 while (!wtw.ReachedWaypoint) {
8 yield return wtw;
9
10 var tgt = new Targetting();
11 yield return tgt;
12
13 if (tgt.FoundTarget) {
14 if (IsNode(tgt.TargetName)) {
15 var wtn = new WalkingToHarvestable(tgt.TargetName, tgt.TargetLocation);
16 while (!wtn.ReachedNode) {
17 yield return wtn;
18 }
19
20 var h = new Harvesting();
21 while (!h.DoneHarvesting) {
22 yield return h;
23 }
24 }
25 }
26 }
27 }
28 }
Notice how simple this is to understand. It reads very procedurally, and yet because this is lazily evaluated, this process can be interrupted at any of the yield points. This control lies with the code that is enumerating through the workflow - that code can act as the gatekeeper, deciding when to get the next work item, when to execute it, when to break out of the loop, what data each bit should have, etc. This can be done in a foreach statement, or I can use the older MoveNext / Current members.
I hope this helps those of you who are still reading to understand how the yield statement can be used to take advantage of the state machine functionality that the C# compiler provides for us. In terms of readability and maintenance, it has proven to be a real boon for me. I hope this has helped you to find the same benefit.
Edit: Here's the disassembled MoveNext() method from Reflector - I haven't cleaned it up a bit. Lots of red squiglies in this one...
1 private bool MoveNext()
2 {
3 bool CS$4$0002;
4 switch (this.<>1__state)
5 {
6 case 0:
7 this.<>1__state = -1;
8 this.<>2__current = new Initialize();
9 this.<>1__state = 1;
10 return true;
11
12 case 1:
13 this.<>1__state = -1;
14 goto Label_01CB;
15
16 case 2:
17 goto Label_00B4;
18
19 case 3:
20 goto Label_00E0;
21
22 case 4:
23 goto Label_0159;
24
25 case 5:
26 goto Label_0198;
27
28 default:
29 return false;
30 }
31 Label_01CB:
32 CS$4$0002 = true;
33 this.<wp>5__1 = this.<>4__this.GetNextWaypoint();
34 this.<wtw>5__2 = new WalkingToWaypoint(this.<wp>5__1);
35 while (!this.<wtw>5__2.ReachedWaypoint)
36 {
37 this.<>2__current = this.<wtw>5__2;
38 this.<>1__state = 2;
39 return true;
40 Label_00B4:
41 this.<>1__state = -1;
42 this.<tgt>5__3 = new Targetting();
43 this.<>2__current = this.<tgt>5__3;
44 this.<>1__state = 3;
45 return true;
46 Label_00E0:
47 this.<>1__state = -1;
48 if (this.<tgt>5__3.FoundTarget && this.<>4__this.IsNode(this.<tgt>5__3.TargetName))
49 {
50 this.<wtn>5__4 = new WalkingToHarvestable(this.<tgt>5__3.TargetName, this.<tgt>5__3.TargetLocation);
51 while (!this.<wtn>5__4.ReachedNode)
52 {
53 this.<>2__current = this.<wtn>5__4;
54 this.<>1__state = 4;
55 return true;
56 Label_0159:
57 this.<>1__state = -1;
58 }
59 this.<h>5__5 = new Harvesting();
60 while (!this.<h>5__5.DoneHarvesting)
61 {
62 this.<>2__current = this.<h>5__5;
63 this.<>1__state = 5;
64 return true;
65 Label_0198:
66 this.<>1__state = -1;
67 }
68 }
69 }
70 goto Label_01CB;
71 }
Subscribe to:
Posts (Atom)
Posts
