15 July 2003

This morning I started out trying to run the Swarm simulations on Penguin, but got a TkExtra error. I think this means I have to change some settings to let me open windows dynamically, but I'll have to find a computer person to help me with that. I'm also having problems compiling test files. I think I'll have to ask Sean where the local copy of Swarm is running, so I can modify my paths.

I started compiling a Spanish corpus. This involved some Google searching, resulting in downloading a Spanish Bible translation. A lot of this needs to be cleaned up, including: 1) Accents removed (accented letters replaced with unaccented versions); 2) Re-formatting to one word per line; 3) Duplicates eliminated; 4) Diphthongs handled. I think I'll try this last one by both suggested methods (delete words with diphthongs / reduce clusters to first vowel) and see how they come out.

I figured out how to run my simulations remotely by using ssh -X to enable Xwin.

Added homophony*20 to Observer.m harmonyGraph. This should help illustrate causation, as harmony can now be seen to rise before homophony and fall after. This makes sense in my head and eventually will on paper.

Daily total hours: 6

_______________________________________________________

16 July 2003

This morning I went to an Emergence group at Bryn Mawr. This counts as work. There was a presentation on scale as it relates to multi-agent systems. There was much discussion of granularity of processes.

I'm trying to start on the Hungarian corpus, but am having some formatting errors. Right now Unix sees the file as one string, but emacs does not. I may have to reformat on a PC and port the data over later.

It bothers me that the Hungarian corpus has only 951 words. I feel that, in order to claim robustness of results, we would need to test a larger data set. The original Hungarian corpus was 995, and the bible corpus 1066. I think I may combine the three corpora and run it over the composite word list.

Today I removed the homophony probe from the harmony graph because it was more confusing than helpful and altered the scale.

Daily total hours: 6

__________________________________________________________

17 July 2003

I think something may be wrong with my Hungarian data. The other Hungarian corpora I ran treated (i,e) as neutral vowels, placing them in both the 'front' and 'back' categories. The results I posted yesterday were run under a distribution of ieOU/Iaou. In order for me to get conclusive results, I have to decide which is right. I also need to confirm that the VHFC successfully handles neutral vowels.

I spent all day trying to get a good measure for harmony threshold with respect to neutral vowels:

P(front & 1)*P(front & 2)^len + P(back & 1)*P(back & 2)^len

with neutral vowels: P((front|n) & 1)*P((front|n) & 2)^len + P((back|n) & 1)*P((back|n) & 2)^len >?100

The inherent problem in this is that the added probabilities of harmonic words can exceed 1. I think this is because it counts twice those words with two neutral vowels.

P(f&1)*P(f&2)^avg*P(n)+P(b&1)*P(b&2)^avg*P(n)?

Daily total hours: 5.5

_____________________________________________________

18 July 2003

My agenda today is to finalise the harmony threshold formula for neutral vowels. It must work for languages with no neutral vowels, as well. My last resort will be to add a field for neutral vowels. When I finish this, I will have to re-run all the harmony corpora.

My solution to the neutral vowel problem is now implemented. Not only did I have to change the formula for the harmony threshold, but I had to redesign the algorithm for calculating harmony in words. I added a new field to the program to account for languages like Hungarian, so now the user is prompted to enter neutral vowels before the other classes.

The current incarnation of the threshold formula exploits the nature of neutral vowels. Because neutral vowels do not affect the harmony (or disharmony) of the word that contains them, it is like they are not vowels at all. At least, any information about their interaction with other vowels is irrelevant. Therefore, we can predict harmony levels by taking into account the average number of NON_NEUTRAL syllables in a word. The old formula stood at:

P(front & 1)*P(front & 2)^len-1 + P(back & 1)*P(back & 2)^len-1

Ignoring neutral vowels, the formula is now:

P(front & 1)*P(front & 2)^navg-1 + P(back & 1)*P(back & 2)^navg-1

Where 'navg' is equal to the avgerage number of syllables per word that do not contain neutral vowels.

I think this works. I'm doing tests to determine whether or not this is a useful figure. I'm also wondering if we should use the average syllable length of polysyllabic words only, given that those are the only ones subject to harmony. However, we would have to be consistent and not take frequency data from monosyllables in order to justify this.

NO. The above measure for a harmony threshold is way too low, as it does not contain those words with only neutral vowels, of which there are many. Maybe we could add this figure to P(n&1)*P(n&2). !!Must also delete neutrals from count_first and count_rest!!

I should probably scrap the whole thing and rewrite it in python...

Daily total hours: 6.5

Weekly total hours: 24

___________________________________________________

20 July 2003

P(harm) = (P(f&1)*P((f|n)2)^avg-1)) + (P(b&1)*P((b|n)&2)^avg-1) + (P(n&1)*P((n|f)&2)^avg-1) + (P(n&1)*P((n|b)&2)^avg-1) - (P(n&1)*P(n&2)^avg-1)

VICTORY!

Daily total hours: 1

Weekly total hours: 25