Revamp of the site

So much is going on with kDemons.org!

We are currently revamping the site and creating a new skin to make navigating the site easier and more productive.

We are also in the process of launching a whole new set of websites to be added to the kDemons site. We’re currently in the process of reserving the domain names, creating the websites’ concepts, installing software and adding initial data. I’ll be posting again when we are finished.

All the new websites along with the old ones will be under an umbrella organization, the kOrganization Inc.

The Javascript Reference is the the newest addition to kDemons Projects. The project aims at providing developers with a comprehensive and easy to use, search and utilize reference of the Javascript language. The project’s front page was set up and it is awiting collaboration. Any help is more than welcome.

All you have to do now is remember the kDemons part! We’ve reserved the kdemons.org, kdemons.com, kdemons.net, kdemons.info, and kdemons.co.cc domains so that any of them will redirect you to the main kdemons.org domain. This includes all subdomains and sub pages.

We’ve finally uploaded a complete mirror of the RFC documents to kDemons! That’s 5500+ RFC documents all up and running. You can access any RFC if you have its RFC number from the url “http://www.kdemons.org/index.php/RFC xxxx” where xxxx is the RFC number. For example, RFC 23 can be found at http://www.kdemons.org/index.php/RFC 0023 (note the leading two zeroes).

This concludes the first phase of the RFC knowledge Repository project and marks the beginning of the second phase: styling! The first phase was to upload the RFCs in their original formatting and providing our users with a mirror reference of the RFCs. The second phase is focused on providing a more reader-friendly version of the RFCs (which will in no way be an official version of the document, instead, a more readable one, keeping the original content of the document as it is). This includes formatting tasks like adding header, section and title formatting, related topics and RFCs, RFC links (obseletes, obseleted by, updates, updated by, is also, and see also), and external related topics links.

I’m currently working on some wiki-templates to be used with RFCs for the second phase of the project and I’ll post again when something new comes up.

Sorry for being a little bit late in updating the blog but I’ve been kind of busy lately. See you all soon.

About 4 hours ago, I posted that I have just had uploaded the 100th RFC after a week or more uploading the RFC documents manually. I also said that this rate is not acceptable if we want to upload 5000+ RFCs. Well, 5 minutes ago, 4 hours after that post, I’ve just finished uploading the 499th RFC document!

What happened?

3 hours of coding in MS Visual Studio and an hour of importing XML pages into the wiki is exactly what happened!

I exported some pages to XML from the wiki, looked at the XML output, written a program in C# that will allow me to add RFC files to a list of RFC titles to import into the wiki then allows me to export these text files into an XML format that is readable by the MediWiki software on the site. I started exporting 10 pages or so, fixed some problems with the code (mostly formatting RFC text into valid XML text) and finally it started to work. I’ve uploaded about 400 RFCs so far using that small program and it has been working fine.

Of course it won’t work for anyone who does not have admin right on the wiki, but the concept is the same for all MediWiki based wikis. So, when I create a page to describe the program and upload it’s code, all you’ll have to do is to change some values in an XML config file, name your text files so that the file name equals the title you want for it when it is uploaded (I had to create another small program to rename the RFC text files to become appropriate to be used as titles in kDemons’ wiki), add the text files you want to add and export them to XML, then finally, import them from your Special:Import page into the wiki, and you have all your text files as articles in your wiki then!

The only problem is you’ll have to wait for me to upload the program and its code (It is 3:00

AM around here right now, so I have to at least get some sleep!)

However, here’s a copy-paste version of the code (that’s the best I could do for now!)

RFC Importer Form in Design View in Visual Studio 2005

MainForm.cs — the code file for the main form

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Xml;
using System.IO;
using kDemons.Windows.RFCImporter.Properties;

namespace kDemons.Windows.RFCImporter {
public partial class MainForm : Form {
private string fileFormat = “”;
private string pageFormat = “”;
private long pageId = Settings.Default.StartPageId – 1;
private long revId = Settings.Default.StartRevisionId – 1;

public MainForm() {
InitializeComponent();
StringBuilder builder = new StringBuilder(1084);
builder.Append(“<mediawiki xmlns=\”http://www.mediawiki.org/xml/export-0.3/\” xmlns:xsi=\”http://www.w3.org/2001/XMLSchema-instance\” xsi:schemaLocation=\”http://www.mediawiki.org/xml/export-0.3/ http://www.mediawiki.org/xml/export-0.3.xsd\” version=\”0.3\” xml:lang=\”en\”>\n”);
builder.Append(“<siteinfo>\n”);
builder.Append(“<sitename>Knowledge Demons</sitename>\n”);
builder.Append(“<base>http://www.kdemons.com/index.php/Main_Page</base>\n”);
builder.Append(“<generator>MediaWiki 1.15.0</generator>\n”);
builder.Append(“<case>first-letter</case>\n”);
builder.Append(“<namespaces>\n”);
builder.Append(“<namespace key=\”-2\”>Media</namespace>\n”);
builder.Append(“<namespace key=\”-1\”>Special</namespace>\n”);
builder.Append(“<namespace key=\”0\”/>\n”);
builder.Append(“<namespace key=\”1\”>Talk</namespace>\n”);
builder.Append(“<namespace key=\”2\”>User</namespace>\n”);
builder.Append(“<namespace key=\”3\”>User talk</namespace>\n”);
builder.Append(“<namespace key=\”4\”>kDemons</namespace>\n”);
builder.Append(“<namespace key=\”5\”>kDemons talk</namespace>\n”);
builder.Append(“<namespace key=\”6\”>File</namespace>\n”);
builder.Append(“<namespace key=\”7\”>File talk</namespace>\n”);
builder.Append(“<namespace key=\”8\”>MediaWiki</namespace>\n”);
builder.Append(“<namespace key=\”9\”>MediaWiki talk</namespace>\n”);
builder.Append(“<namespace key=\”10\”>Template</namespace>\n”);
builder.Append(“<namespace key=\”11\”>Template talk</namespace>\n”);
builder.Append(“<namespace key=\”12\”>Help</namespace>\n”);
builder.Append(“<namespace key=\”13\”>Help talk</namespace>\n”);
builder.Append(“<namespace key=\”14\”>Category</namespace>\n”);
builder.Append(“<namespace key=\”15\”>Category talk</namespace>\n”);
builder.Append(“</namespaces>\n”);
builder.Append(“</siteinfo>\n”);
builder.Append(“{0}\n”);
builder.Append(“</mediawiki>”);
fileFormat = builder.ToString();

builder = new StringBuilder(264);
builder.Append(“<page>\n”);
builder.Append(“<title>{0}</title>\n”);
builder.Append(“<id>{1}</id>\n”);
builder.Append(“<revision>\n<id>{2}</id>\n”);
builder.Append(“<timestamp>{3}-{4}-{5}T{6}:{7}:{8}Z</timestamp>\n”);
builder.Append(“<contributor>\n<username>{9}</username>\n<id>{10}</id>\n</contributor>\n”);
builder.Append(“<comment>{11}</comment>\n”);
builder.Append(“<text xml:space=\”preserve\”>\n”);
builder.Append(“&lt;pre&gt;\n{12}\n&lt;/pre&gt;\n[[Category:RFC]]\n”);
builder.Append(“</text>\n</revision>\n</page>\n”);
pageFormat = builder.ToString();
}
private void btnAddRFCs_Click(object sender, EventArgs e) {
if (dlgAddRFCText.ShowDialog() == DialogResult.OK) {
int i = 0;
foreach (string file in dlgAddRFCText.FileNames) {
if (!lstRFCs.Items.Contains(file)) {
lstRFCs.Items.Add(file); i++;
}
}
if (i > 0) { MessageBox.Show(i.ToString() + ” RFC files added!”); }
lblItemsCount.Text = lstRFCs.Items.Count.ToString();
btnRemoveRFCs.Enabled = btnExport.Enabled = (lstRFCs.Items.Count > 0);
}
}
private void btnRemoveRFCs_Click(object sender, EventArgs e) {
if (lstRFCs.SelectedItems != null && lstRFCs.SelectedItems.Count > 0) {
int i = 0, max = lstRFCs.SelectedItems.Count;
for (int j = 0; j < max; j++) {
lstRFCs.Items.Remove(lstRFCs.SelectedItems[0]); i++;
}
if (i > 0) { MessageBox.Show(i.ToString() + ” RFC files removed!”); }
lblItemsCount.Text = lstRFCs.Items.Count.ToString();
btnRemoveRFCs.Enabled = btnExport.Enabled = (lstRFCs.Items.Count > 0);
}
}

private void btnExport_Click(object sender, EventArgs e) {
if (dlgSaveXMLFile.ShowDialog() == DialogResult.OK) {
try {
StringBuilder pages = new StringBuilder(50000);
foreach (object f in lstRFCs.Items) {
StreamReader reader = new StreamReader(f.ToString());
string rfc = reader.ReadToEnd(); reader.Close(); reader.Dispose();
rfc = rfc.Replace(”“, “”);
rfc = rfc.Replace(“”, “”);
rfc = rfc.Replace(“&”, “&amp;”);
rfc = rfc.Replace(“<”, “&lt;”);
rfc = rfc.Replace(“>”, “&gt;”);
rfc = rfc.Replace(“\”", “&quot;”);
pages.Append(makePage(rfc, f.ToString()));
}
string contents = string.Format(fileFormat, pages.ToString());
StreamWriter writer = new StreamWriter(dlgSaveXMLFile.FileName);
writer.Write(contents);
writer.Flush();
writer.Close();
MessageBox.Show(lstRFCs.Items.Count.ToString() + ” RFCs were imported successfully!”);
}
catch (Exception ex) {
MessageBox.Show(ex.Message);
}
}
}

private string makePage(string rfcContent, string rfcFileName) {
pageId++; revId++;
return string.Format(pageFormat, Path.GetFileNameWithoutExtension(rfcFileName),
pageId, revId, DateTime.Now.Year, DateTime.Now.Month, DateTime.Now.Day,
DateTime.Now.Hour, DateTime.Now.Minute, DateTime.Now.Second,
Settings.Default.AuthorUserName, Settings.Default.AuthorId,
Settings.Default.SummaryComment, rfcContent);
}
}
}

application config file

<?xml version=”1.0″ encoding=”utf-8″ ?>
<configuration>
<configSections>
<sectionGroup name=”applicationSettings” type=”System.Configuration.ApplicationSettingsGroup, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ >
<section name=”kDemons.Windows.RFCImporter.Properties.Settings” type=”System.Configuration.ClientSettingsSection, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ requirePermission=”false” />
</sectionGroup>
</configSections>
<applicationSettings>
<kDemons.Windows.RFCImporter.Properties.Settings>
<setting name=”StartPageId” serializeAs=”String”>
<value>5000</value>
</setting>
<setting name=”StartRevisionId” serializeAs=”String”>
<value>5000</value>
</setting>
<setting name=”AuthorUserName” serializeAs=”String”>
<value>VC</value>
</setting>
<setting name=”AuthorId” serializeAs=”String”>
<value>2</value>
</setting>
<setting name=”SummaryComment” serializeAs=”String”>
<value>Initial import of RFC document Contents</value>
</setting>
</kDemons.Windows.RFCImporter.Properties.Settings>
</applicationSettings>
</configuration>

2 minutes ago, I finally uploaded the RFC 100 document to kDemons.com’s wiki!

Only around 5000+ remaining.

At this rate, it will only require another 500-700 days to complete the whole library, which by all means is not acceptable.

The good news is that I’m only uploading the documents manually while I develop a small program to export the text files into XML that can be imported to the site in bulk! The problem is that I’m not finding the time to do so (not only the real time, but time when you can write code!)

If and when the program is completed, I’ll publish it as open source code and at the same time it will most probably be a good sample of file system and XML handling in .NET (C#)

I’ll post more news when there are more news!

Finally, a page that shows the site map of the kDemons wiki section was created. You can view the sitemap at: http://www.kdemons.com/index.php/kDemons:Sitemap

It’s not much, but it is a start for stabilizing the categorization for now at what it is! It will be hard to add to the current form of the sitemap when new categories are added, but not impossible. However, because editing the sitemap is a complex sensitive task, the page was protected so that only experienced editors can edit it (hoping that even experienced editors won’t make a miss of it as it is right now!

I’ll be creating a more brief template for the sitemap to be included in other pages and templates when I have the time.

The first project of kDemons to start implementing and that is actually related to kDemons’ contents is finally here!

I have started a project to import all RFC documents (the defining documents of the internet) so that they can be published yet for another time. However, with the wiki capabilities of the site, I hope that more than just republishing the RFCs is going to be happening.

For experienced users and contributors who already understand the RFCs or some of them, we can start writing some explanations to accompany the RFCs themselves. The formatting of the RFCs can be fancied up a little and finaly, graphs and the like could be created for the documents.

The project aims at re-publishing a “pretty”, readable and understandable version of the RFCs. If this succeeds, all contents will be initially imported from the RFC Editor site as plain text document added into <pre> tags in kDemons pages. Each RFC will be imported into one kDemons page. Currently, if anyone would like to add some comments explaining the RFC, it should be added to a sub-page of the RFC’s page named “About”.

The category for all RFC documents is RFC. Still working on some templates to add description and meta data to resources in general and RFCs in specific, I’l post when something new comes up in that area. Salute for now!

N.B. I’ve contacted the RFC Editor site asking about the license of their downloads to check for any future problems that might arise and to ask for permission to redistribute the documents. I’m still waiting for their response, and I’ll update this post when one arrives!

• Regarding categorization, a templates category tree has been created to allow categorization of templates. templates themselves are categorized and they might add categories to the page they are used in, so do not confuse the two aspects! For example, the {{note}} template adds the page it is used upon to the Pages With Notes category. The note template itself is within the Editing Utility Templates category.
• As for templates, the license templates have been updated with new looks and easier to include in articles and pages. They have been categorized, and the feature to add pages they are used on to the corresponding category was added.
• The templates summary page was also updated so now it includes working samples of the templates and examples of their usage.
• License text was added for all the GNU License agreements and for the creative commons CC-BY license and the corresponding license template was updated to point to the kDemons page for the license text instead.

I have created a proposed categorization hive for kDemons wiki-pages and added categories for the kDemons namespace pages. You can review the current categorization hive here. If you have any ideas, suggestions or modifications, please feel free to change to the listing (please do not create categories yet however).