Can someone explain the Natural Actor Critic algorithm to me in simpler terms?

Ruzihm · 2012-05-25T05:02:21.032963+00:00

To really understand this paper, you're going to need not only the reinforcement jargon (the Sutton and Barto book that syyra linked to will, for the most part, handle that) but understand natural gradient methods (which are a fair bit of brainfuck). I wouldn't recommend starting with Amari's paper, you'll just end up more confused. References 10 and 11 seem like the most closely related work and #10 is a journal paper which will hopefully provide enough relevant details to bootstrap from. #5 sounds like lecture notes which don't appear to be easy to find, but if you have access through a university, this is the link.

lvomrm · 2012-05-25T03:55:27.375080+00:00

for example using object-oriented languages to have nodes as objects (which can act more independently)

Good god, no. This would be slow as molasses. Even if implementing it in C, you really want to take advantage of highly optimized matrix multiplies.

The thing about neural networks is that in the course of training you need to evaluate both the forward and backward pass thousands to millions of times, depending on the size of the network, complexity of the task, and amount of training data. Speed counts, and the overhead of even the leanest object-oriented implementations will be far less performant than one implementing the summations for all hidden units/output units on a mini-batch of examples with a highly-tuned GEMM. The difference between a good GEMM and a bad GEMM can be significant, even on a single core; on multiple cores it can be staggering.

lvomrm · 2012-05-24T23:20:02.113647+00:00

As others have said, a fast neural network implementation really just means an optimized BLAS and good reuse of allocated buffers. There is nothing really programming language specific about it, though it will be harder to eek out speed in higher level languages without help from lower level libraries.

If you're interested in the computational considerations involved in fast implementations of feed forward networks, there's this paper from the NIPS deep learning workshop. There's also a paper at ICML 2012 called "Utilizing Static Analysis and Code Generation to Accelerate Neural Networks" but so far it doesn't look like a preprint is available.

dr_chickolas · 1 day

However, I don't think there is a requirement for kernel regression to have the basis functions centred on each training point - you can have as many or as few as you want. In this respect, I think the difference really boils down to the weighting, which is present in RBF networks, but not in kernel regression.

Usually those points are centered on training examples, though: the only reason not to use all of them would be the computational cost. RBF networks adapt the placement of the centers of the basis functions by gradient descent/backpropagation; they also adapt the bandwidth of each kernel individually. They are notoriously hard to train well, though: the form of the objective function makes the error surface with respect to the parameters a pain in the ass to optimize over.

Wouldn't surprise me if splines were more efficient in one or two dimensions, but I think the way they are parameterized would make for trouble in higher dimensions (i.e. a number of parameters that grows superlinearly with dimension).

dr_chickolas · 1 day

As far as I can see, "kernel regression" typically refers to local mean or local linear regression, where the influence of each training point is controlled by a kernel function, such as a Gaussian, centred on that point.

Basically. It is a non-parametric method, meaning the complexity of your estimator grows relative to the amount of training data you have. You can see this from the basic formula for the Nadaraya-Watson estimator. Also, all points get equal weight in the estimate.

RBF networks are just a particular type of "kernel regression"

Not quite. RBF networks are a parametric method, whereby you fit the parameters of a set of K radial basis functions, which are then linearly combined in a weighted combination to obtain an estimate of the target. Whereas at test time, kernel regression will compute the RBF for the distance between every training point and the test point, an RBF network will compute the kernel for the distance between the test point and K arbitrary center-vectors, where those center vectors are adapted during training. Usually the kernel width is also adapted. You could, of course, have an RBF network the size of your entire training set and make those your center-vectors, fix the kernel width across all of the basis functions, and set all of the weights to 1/N_training_examples, and you'd end up with non-parametric kernel regression; RBF networks are thus a more general case.

Last, splines use polynomial basis functions, which are zero below their origin, so they are asymmetric (in contrast to RBFs and usually to kernel regression).

Typically, you fit a polynomial interpolant of a given order between any two knot points, with additional constraints: i.e. if you have f_1 interpolating between knots a and b and f_2 interpolating between knots b and c, then typically df_1 (x) / dx evaluated at b will be constrained to be equal to df_2 (x) / dx, and usually (for cubic splines) d² f_1 (x) / dx² and d² f_2 (x) / dx² evaluated at b should also be equal. This achieves smoothness around the knot points, other penalties may also be applied.

By 1-d scatter plot smoothing, you mean you have a single scalar value 'x' for each point of interest and you want to produce a smoothed "density" estimate d(x) indicating relative concentrations along the line? I think kernel regression should be just fine for that. Splines are more for interpolation where you have an (x, y) pair at each training point.

cheapwowgold4u · 1 day

Database access is tremendously expensive

Truly, a cancer of the Internet age.

Ririii · 2 days

By multicultural free-for-all I meant "come here and don't learn the language or assimilate into the culture or society at all, do whatever you like". If the 911 services were available in her language, she sure as hell didn't know about it. Possibly she didn't even know the number, as she handed me the phone.

One could argue that you could still have "no one minority on any better footing than anyone else" if we all just standardized on English. Then, I guess native speakers would be on better footing, but native speakers are not uniformly members of a particular minority.

combatllama · 2 days

most parents pay for their kids tuition

In what universe?

Ririii · 3 days

force every business to speak your language even if they're in an area that doesn't use it.

No one's saying they can't use other languages in addition to English. But this is not Somalia; opening a business here and running it however you like is not an absolute right.

One obtains a City of Toronto business license at the pleasure of the municipal government, acting on behalf of the people of the City of Toronto, the majority of whom are English speakers, either as their first language or an adopted one. Modulo the specific wares you peddle, your business should be ready to serve any resident of the city of Toronto at least in the prevailing tongue of commerce. If you want to serve individual communities, including your own ethnic community, in other languages, by all means, do so! But there ought to be at least one person on the premises who can conduct business in English at any given time.

Even setting aside the rights of customers, building safety inspectors, food inspectors, tax auditors, and emergency personnel (police, fire, paramedics) all need to be able to communicate with the proprietors and staff of any public-serving establishment. Forgive me for thinking that the burden should lie with the proprietor, not with the city, to surmount the language barrier.

Ririii · 3 days

So given that and using your language logic every Quebecois should speak english.

If you think that "country" is the relevant level of granularity for such an assertion. Which, especially in a country as big as Canada, and with a history such as that of Canada's, is frankly absurd. When you come to a region you should be prepared to learn the language of the majority. Like I said, even entire provinces are units far too big to have a uniform language policy, and laws in both Ontario and Quebec account for this.

No matter who was here first the English defeated you

"You"? Even ignoring the dozens of generations that have passed since the French and Indian War, my family isn't of French extraction. My earliest ancestors ended up here when the British were forcibly starving an entirely different ethnic group overseas. Keep your bigotry to yourself.

and that defeat has stuck to this day.

It wouldn't have without a great deal of complicity on the part of the French upper classes. The British granted the French locals a good deal of autonomy and self governance from the very start, namely because there were too damned many of them to handle without the support of the local aristocracy and clergy.

but that is the way things go.

If you're implying that that means anything to present-day Quebec, particularly as concerns language, the Supreme Court of Canada disagrees.

Ririii · 3 days

They lack the single most critical skill for a service industry job. You wouldn't hire someone without a driver's license for a delivery man's job; speaking the language of the land is a job skill like any other.

I would say it's the height of selfishness to come to a place where a certain language is spoken by the majority and expect everyone else to accommodate your ignorance of that language. However, I will readily admit that the vast majority of immigrants to Canada do not fit this description.

Ririii · 3 days

I'd agree, to a point, except that by making the decision to come to this country they are implicitly taking on certain responsibilities: following our laws, paying taxes, and learning the language. The burden is, and should be, on newcomers to adapt to their new surroundings, since they're the ones who chose this life. I'd say the same of Canadians living abroad.

That's not to say that we can't stand to be more welcoming and accommodating while they learn, though I disagree with your comment about the police. The notorious Polish guy at Vancouver airport was extremely unfortunate but he was acting kind of nuts. The cops went too far, but I wouldn't have wanted to be in their shoes dealing with an unstable individual with whom they could not communicate.

Ririii · 3 days

By all means protect the French language, but for safeties sake, if there's a potential hazard on the road ahead, signs in English would be a very reasonable precaution.

Sure, I agree completely that there are certain small aspects that go too far. The fact that I can't order a certain air conditioner online from Future Shop because it doesn't contain French instructions is ludicrous. The fact that two francophones can't choose to send their child to an English public school is kind of nuts. I'd like to see more of the kind of thing you're talking about (highway signs, mainly) -- safety instructions in English exist on the Montreal Metro, so it's not completely out of the realm of discussion. All in all, though, I think the language laws are reasonable and accomplish their goal.

Ririii · 3 days

The federal government is officially bilingual, so their services will likely be available in English and French in BC.
Packaged food is federally regulated and thus the bilingual labels. I'm not sure if this is required or not, but it might be convenient to not have multiple versions of a product that need to be inspected separately.
The transportation sector (interprovincial and international) is federally regulated, thus the bilingual announcements on Via Rail, Air Canada, etc. This makes perfect sense if you hope to such services to accommodate all Canadians as they travel throughout the country, and would make good business sense even if not for the law.
Finally, there is also a small but significant francophone minority (according to this site, 7% of the population is conversant, possibly immigrants from other countries where French is a second tongue; around 2% are mother-tongue francophone. The BC government may have chosen to accommodate the francophone population in other ways I'm not aware of.

That said, I've been to Vancouver several times (once with a lot of francophones, it turns out) and I didn't encounter anything French save for packaged food labels.

Ririii · 3 days

It's not an unreasonable request to ask that people working in a position where they interact with the public have some passable skills at speaking the de facto language of commerce. In most places, such a thing is so obvious as to not even merit a mention. In "multicultural mosaic" Toronto, on the other hand... The fact that we have to put up bilingual "NO DUMPING" signs near Spadina and Dundas (not in the two official languages of the country, mind you -- in English and Chinese) speaks volumes about how such accommodations can go too far.

I'm not saying that businesses shouldn't be able to serve you in whatever languages they like. Lots of banks in Toronto do good business in Italian, Portuguese, Mandarin, Cantonese, Vietnamese. I'm saying that a business open to the public should at least be able to serve its clients in the prevailing tongue of the region -- that is to say, English. At the very least they should be assimilated enough (yes, "assimilate" is not a dirty word) to be able to respond sensibly in an emergency.

Ririii · 3 days

French is the language of the majority in Quebec. Always has been. And if you're implying that English is the language of a "majority of Canadians", a) Canada is too large for that kind of argument to make sense (even provinces are too large, hence why you can get access to Ontario government services in French in certain communities and not others -- there are also legally bilingual cities and towns in Quebec), b) since 1969, the federal government has been officially bilingual, giving equal legal status at the federal level to French and English, c) the Supreme Court has upheld the authority of the Quebec provincial government and the constitutionality of most of the articles of the Charter of the French Language, with certain ones having been amended over the years.

I'm surprised this needs explaining. Also, if you want to get technical, the francophones were here first.

Ririii · 3 days

No one's suggesting they shouldn't work. I'm saying they shouldn't work serving the public.

You can find work as a unilingual anglophone in Quebec, too. Just not in the service industry. For that, you need to at least speak French. Bonus points if you're bilingual.

Ririii · 3 days

That's great. Really, honestly and truly, I'm happy that we can offer those services so widely.

Nevertheless, I illustrated a situation that was a profound failure of the multicultural free-for-all. Either this woman (and the other woman with her) did not think she could speak to the 911 operator adequately, or she didn't know that a Korean-speaking operator would be available (would they? you said 311, not 911), or she was so culturally isolated that she didn't even know the damned phone number to call. It was an incredibly hot Sunday afternoon, a heat advisory was in effect, not a lot of people were around.

Relying on someone else to be able to relay the situation meant relying on me, and it could've been whole, critical minutes before I happened to walk in.

Ririii · 3 days

I did. I walked in, he was already on the floor, and they rushed over to me with a phone. I attempted CPR as well. I have no idea how long he'd been there, though.

TestZero · 3 days

The disabled bathroom where I used to work had a strip you could push for assistance and it was labeled: Use only in "emergency". Whenever I read that I imagined the guy printing it thinking "Stupid cripples! They could walk if they really wanted to."

Ririii · 4 days

Also, I believe she is calling herself Quebecoise because she geographically from Quebec, not in an ethnic way.

And herein lies one of the fun idiosyncrasies of Quebec society. People throw around those demonyms quite freely but are always a little ambiguous about what they actually mean. Then when the xenophobic element in the modern PQ charges up their identity politics schtick they have to dance around the borderline racist stuff and pretend that "québécois" means something other than "white, old-stock, French descent". It's kind of a fun-to-watch trapeze act.

Ririii · 4 days

My favorite thing about my nation is that we don't have an inferiority complex towards English Canada, we do not care what you think of us.

People who go out of their way to proclaim their lack of inferiority complex, well... you can see where this is going.

Ririii · 4 days

It's certainly how I feel about the way the Conservatives and Liberals treated it, how they attack the NDP policy (whether that's fair or not I'm not particularly able to say) and how far too many commentors online act too.

The NDP are notorious flip-floppers on the issue of both language laws and the national question. Layton has publicly stated his support for the Clarity Act but was also behind the Sherbrooke Declaration, and yet the two are completely irreconcilable. Not only does it contravene the Clarity Act but according to the Supreme Court of Canada's "Reference re: secession", the Sherbrooke Declaration arguably flies in the face of the Constitution itself.

I think attacking the NDP position on Quebec is fine because it's very easy to demonstrate that they don't have a coherent position.

Ririii · 4 days

Wow, that collection of letters to the Post shows a lot of shameful misinformation, especially re: equalization payments. Quebec has traditionally received the highest absolute amounts in equalization payments, but their population traditionally outnumbered all the other have-not provinces put together. Per capita the Maritime provinces have always been the real money sink.

And, as an Toronto-born anglo-Canadian living in Montreal, I'm totally fine with the language laws. You come to a place, you learn the language of the majority; it's not rocket science. I think Toronto could learn a thing or two from them, i.e. mandating that shopkeepers be able to serve you in the majority language of the land. (I've personally witnessed a man die of heat stroke in a convenience store who might've been saved if the Korean shopkeeper had spoken enough English to feel confident dialing 911.)

LoopyDood · 4 days

Knight in the center increases your horsepower!

dwf

TROPHY CASE

Four-Year Club	reddit mold
Verified Email

dwf

TROPHY CASE

you'll need to login or register to do that

create a new account

login