Home » Technical issues

# Category Archives: Technical issues

## A long step towards the adult PhysicsOverflow

Today, the new category system, which allows an unrestricted number of category-levels, has been installed on PhysicsOverflow. This system was required to enable the categorization of submissions beyond the four levels provided by the actual Question2Answer (Q2A) framework. This new system is almost invisible for the user, but has the important function to make a (future) large number of submissions searchable by the user. About two and a half months have been required to develop this system, you had to wait for a long time. Therefore I like to give you some insights.

The original category system of the Q2A framework is a hierarchical database model. Every node (a category) is linked to its parent by the ID of the parent node. Such a model enables a fast writing of new nodes (you just create the node and link it to its parent), while queries through the tree are usually slow. Q2A solved this issue by a fixed number of 4 category levels. To be able to find parents in a short time, the path to the parent is hard-coded in every post. This means that every post contains four indices storing the way back through the tree. This is a clever and fast solution. However, it can not be extended to an unrestricted number of categories, it even gets slow if extended to 8 levels for instance. The category system is written in the core code of Q2A and spread over a large part of the system.

Another issue arising when we increase the number of category levels is the user interface. Actually, when the user asks a question, he has to select the category for this post. The user interface uses select tags like:

It is clear that for instance for eight categories, the place on the page is too small to display all these tags side by side. Therefore also the user interface had to be changed. The new user interface take much less room and looks like this:

For the new category system, I have implemented another database model, called nested set model. It allows for an unrestricted number of category levels. While queries through the categories become very fast, the insertion of nodes is slow, because all indices of the whole tree have to be changed. However, changes on the categories will be used much less frequent than for instance to display the tree. Like this, it is well adapted to our needs.

As already mentioned, the original category system is placed in the core code of Q2A and is active in almost all pages provided by the framework (even in pages I never expected this). I had to replace 9 files completely, all select specifications for database accesses to the category system had to be renewed and, naturally, the code for the nested set model had to be written. To give you an impression on the size of the task, here some numbers: The original Question2Answer framework consists of about 36’000 lines of code. Until today, I have written 18’100 new lines of code for PhysicsOverflow in form of plugins, layers, overrides and changes in the core code, about half of the size of the system. The new system required 5’800 additional lines of code, which explains, why it took that long to realize it. I hope now that I was able to test all use-cases so that there remain eventually only minor bugs.

The next development step will be a surprise, stay tuned!

polarkernel

## PhysicsOverflow is Living and Animated

Wondering what happened on PhysicsOverflow while I have developed the preliminary phase of PhysicsOverflows reviews section, Reviews I, I have looked on the host providers statistic and have done some queries in the event log of our database. Here are some numbers illustrating the pleasant activity on our site:

Number of visits in May 2014 (provider’s statistic):

As you can see, the number is slightly growing. The number of visits has been above 300 all the time approaching now 400 by end of the month. Maybe somebody has an explanation for the peak after the second weekend?

In the event log table of the database, 14’482 events have been logged in May 2014. Some interesting numbers are

• 4126 times, a user has logged in.
• 200 questions have been written.
• 353 answers have been posted.
• 4995 upvotes have been given.
• 86 new users have registered on the site.

I think this is really not bad for the second month after publishing the site. Maybe we will already have quite a number of submissions in the reviews section by end of next month? Interested? Submit a paper to PhysicsOverflow!

## Reviews I: complete

We have decided to launch the reviews section earlier, an its preliminary phase, Reviews I.

We don’t yet have all the fancy features that will come in Reviews II, such as the automatic paper imports, the fancy paper sorting system, and so on and so forth, but the preliminary version is here! So, how does Reviews I work?

Basically, since the plugins for mass-importing, daily importing, and importing by users aren’t ready yet, only administrators can currently create submissions. Users may request the submission of any paper for review, and claim authorship to a paper in the following threads respectively:

Submission Creation Requests

Claims of Paper Authorship

## If an administrator can create a submission, why didn’t you just open this feature up to everyone?

Because the plugin isn’t perfect yet.

The main issue I see so far is that the plugin allows you to submit as submission to any section, including Q&A, Reviews, Meta, and even the section Closed Questions.

In the final version of the plugin that will eventually be available to all users, the plugin will only ask for the ArXiV paper ID, e.g. “1403.1254”, and other things like Paper title, Date of submission, paper URL, etc., will be automatically found by the plugin itself. This also prevents users from misusing the plugin to fake the date at which the paper was submitted, and so on. The plugin should also verify that a paper with the submitted paper ID exists. And even if people don’t misuse it, we should be able to advertise the reviews section as “Just enter the ArXiV ID to submit your paper.” : )

## What next?

• Mass-import all physics papers from the ArXiV.
• Make a bot which imports all new physics papers on the ArXiV daily.
• Simplify and secure the plugin as mentioned above.
• Increase the maximum category depth to allow for a hierarchial tag system.
• Start mass-retagging all the papers. I’d stick to “mass-retag all new papers, and twice the number of old papers as new papers everyday”. Sorry Ron, but I don’t think it’s practical to rush through the entire ArXiV and put it in an elaborate tag system in a few hours. The community needs to be told “Retag every paper you see that you can.”.

And then there are plans for continuous development, including locally hosting contributions, locally editing contributions with a full-fledged TeX editor like ShareLaTeX.com, academic advertising of papers to pay for hosting, and so on.

So what are you waiting for? Submit a paper to PhysicsOverflow!

## Status of the Reviews section

About a month and a week after we went public, PolarKernel has finished the first phase of the reviews section, all through a plugin, without touching a single line of Q2A core.

The “submission”-type post is nearly ready in that we now have a post type which is similar to that of a question but there are two voting criteria and the ability to add multiple authors. This is attached to a review type which is similar to the type of an answer but is called a review instead.

### What now?

This was the first phase (out of four) of the development of the reviews section. This is probably the longest phase besides the fourth phase which is the ongoing development of the feature, including local hosting of contributions, and a TeX editor for local editing of contributions. The second phase of the reviews section is a plugin for the mass-creation of papers and another for the daily import of papers, and yet another for the manual import of papers.

I will now leave you with an amazing screenshot of the reviews section. This is a screenshot from a running offline site, not a mockup! (we know the styles aren’t perfect; we’ll fix them; don’t worry)

Screenshot of submission type and review type.

## We Have Liftoff!

Liftoff

PhysicsOverflow public beta is online now since 17 days. After a turbulent start with some database issues on our host and spam attacks, now the site is stable and working fine. After an ongoing and continuous improvement of details, the section with Q&A on physics takes now the main part of activity on the site. Since we have gone online at 4th April, more than 150 new users have registered on the site and many of them are already quite active. The number of visits per day exceeded the number of visits on Theoretic Physics on SE since the first day, as can be seen in the following graphics:

This is great, since we aren’t even part of a huge network! Our questions per day is about 5.6 (excluding imported posts), which is a lot more than TP.SEs. However, as already stated, the site does not depend on these figures, there is no deadline, as it was on TP. Also the term beta does not mean that the site will go away, it means only that we have still much more ideas to obtain the full-fledged version of PhysicsOverflow.

Now is the time where the site gets shaped. If you participate now with your votes, ideas, opinions, questions and answers, you contribute to build a site with contours as you like them. Don’t stay outside, have a look at PhysicsOverflow, register there if you like it, or contribute here on this blog.

Review Section

We are still a small team. However, we are working with full power to be able to leave the beta state and to complete the site, as intended. The main part for this will be the Review Section, as already announced in this blog. This is not only a reconfiguration of the site, it requires a considerable part of new development, because such a feature is not foreseen in the Question2Answer framework. In detail, the following main functionalities have to be developed:

• Integration of new pages, called Submission and Review.
• Two voting criterias, one for originality and one for accuracy.
• New page design, enabling display of a score value from the votings.
• Redesign of voting mechanism, distributing votes to multiple authors.
• Integration in the rep update and recount system.
• Adding and managing the required database tables.
• Increasing the category depth to realize hierarchical tagging system.
• Integration of score as new sorting criterion.
• Software for mass import from ArXiV.
• Software for daily import from ArXiV.

We are on the way with all that and look forward to realize these steps within a reasonable time. Stay tuned!

## Technical problems with the mySQL data base server :-(

Dear PhysicsOverflow community,

today we seem to face (for the first time within a time period of about  two months) some problems with the mySQL data base server on the host side. This lead to a short down time of PhysicsOverflow of about 10 min some hours ago, and to a new longer down time of an hour just now. This is very annoying, in particular as we have just started our public beta …

So we apologize for this inconvenience, but all we could to is complaining to the administrators of our hosting provider, which is what Polarkernel did:

Hi
There seem to bee problems with the database on our host. Sorry, but I can’t change it. See here the transcript of my contact with our host provider (translated by Google):
Now talk to ‘Elodie’
You: I got today on my site “www.physicsoverflow.org” for the second time a “database connect error”. Can you figure out what’s going on?
Elodie: Yes, actually, there are some difficulties with the MySQL Server today.
Elodie: The technicians are already aware and working on it.
You: OK, I hope the issue can be resolved soon. I have opened my site tonight and it’s a shame, especially when such errors occur at the beginning.
Elodie: I see. I do not think it takes a long time. But at present we have not yet received any specific information from the technicians. Except to the confirmation that they know it and work on it.
You: OK, then I’ll wait. Thank you!
Elodie: No cause 🙂
polarkernel
Now PhysicsOverflow seems to work again for now
but new down times can not be excluded until the technicians of our host state that they have successfully resolved the issue …
We will report here as well as on the site itself as soon as everything is ok again.
But lets not hope that the worst is over now, I am so excited and happy about all the great nice wise people joining in so far 🙂
Cheers

## Physics Overflow is going to go public!

Finally, a long-awaited announcement. Physics Overflow’s private beta will end next week at midnight on Thursday, i.e. at 0000 hours, 4 April 2014!

This means that the public beta, which is the fifth phase in the history of Physics Overflow is going to begin. The phases so far are:

• The blog
• The offline site [Foetushood]
• The private beta [Infanthood]
• The public beta (will begin next week) [Childhood]

So Physics Overflow will reach it’s childhood at the age of one and a half months next week! Sorry, couldn’t resist from giving stupid analogies.

Seriously, these things remain to be done, with the deadline (I copy an answer of mine from Physics Overflow):

I think that the TP.SE question Public beta: Attracting users? is also relevant to us.

In my opinion, there are some plug-ins (a job for polarkernel) remaining to be installed as soon as possible, such as

These plug-ins are good to have, and the first one is almost a necessity. I wonder why Q2A does not incorporate it in it’s core.

The second one gives admins some great tools to see how well the site is faring at a glance. It could even be made public I hope.

Additionally, we also may want to decide on solutions for having a chat room. That somehow reminds me; we may also consider retaining the tpproposal blog as a (hopefully $\infty$) life-long blog for Physics Overflow. The chat room; I’m not sure if we should wait for the public beta to start or not, but I just thought of reminding everyone about it.

From the things you have listed the following need to be done by 0000 hours, [CEST]

• Sunday:
• Tuesday:
• We should invite everyone listed in the blog post.
• We should announce on the TRF thread and invite Lubos too. Dilaton will write a guest post on TRF.
• We should obviously announce on the blog.
• On the blog, user10001 (dushya) had the idea of promoting the site to various physicists in the world.
• Wednesday:
• I (and anyone else who is interested) need to finish 14 and 15 of the FAQ.
• The long users need to be corrected (I will do this).
• The main page needs to be set to Q&A.
• Thursday:
• Change permissions
• Edit robots.txt (a job for polarkernel)