cdap application names need to support "."
Description
Release Notes
Activity
Tommy CarpenterSeptember 17, 2016 at 11:38 AM
We are not using URLs as IDs. There are no "/" or "&". My original question was worded incorrectly, my appologies to you, I have corrected it.
Tommy CarpenterSeptember 17, 2016 at 11:34 AMEdited
Not sure what else I can say here. Low level implementation details (like "." is not allowed in Hive) is making their way all the way up to your REST APIs, the public interface to the world. This seems like "lazy" API design, instead of providing another layer that abstracts these issues away. I guess the issue isn't "application names need to support "." as your REST APIs do not support it, whereas underneath you could translate the name and maintain a mapping for internal use.
Tommy CarpenterSeptember 17, 2016 at 10:28 AMEdited
Sorry, I misspoke when I talked about URL. The original title of my question and my original post captures what I want.
The fact that Hive does not allow "." is irrelevent.
We have now had to wrap your APIs with a layer above that maintains a mapping from "whatever.whatever" to "whatever_whatever" for example. You could do the same. The "." would never been seen by your existing APIs.
But you may have many other clients who need to do the same thing. The consul example I gave is very common – to want CDAP applications to be identifiable by a DNS name. So any of your clients who want to do this may have to wrap your API.
I see no reason that you could not provide a wrapping layer. It would not force the client to re-wrap (I didn't understand @Andreas Neumann comment on that).
Andreas NeumannSeptember 17, 2016 at 4:51 AM
Hi Tommy, in the subject you were requesting to allow ".", but in the comments you said that you would like full URLs as app names
> I need to be able to deploy an application to CDAP where the application name is a URL.
I am not sure whether I misunderstood, but a full URL would contain / and ? and & etc.
Dots would also pose a problem because they are not allowed in, for example, Hive database or table names.
As I said, from the CDAP perspective, these are really names, and it is not that uncommon to restrict what is allowed as part of a name - I already named a few examples.
NitinMSeptember 17, 2016 at 2:43 AM
@Tommy Carpenter Your leads are correct that in a URL '.' (dot) is not considered as special character. But, the point that you are missing is that you guys are constructing URL and using URL as id. In CDAP, the App Id (/apps/<app-name>) has restrictions in terms on characters. RFC you are referring makes sense for URL, but not so much as Application Id.
We are using CDAP inside a system where component (mircorservice) names contain ".".
However, I get:
Deploying application by hitting: http://135.207.160.159:10000/v3/namespaces/default/apps/fff70fa0f1eb45c1ae143335b35940c4.hello_world_cdap_app.service_id.location_id.att.com
2016-09-15T18:23:17 LOG <test_cl_wagonized_1> [cdap_app_a6a3c.start] INFO: Reponse code: 400, Response: Invalid app name 'fff70fa0f1eb45c1ae143335b35940c4.hello_world_cdap_app.service_id.location_id.att.com': Invalid Application ID.
If I try to replace the "." with underscores, it works:
'fff70fa0f1eb45c1ae143335b35940c4_hello_world_cdap_app_service_id._ocation_id_att_com'
Is valid and gets deployed.
Perhaps it is worth explaining why we want to do this so it does not seem like nitpicking for the sake of doing so.
We are now bridging CDAP applications with other types of containers running on external clusters, e.g., Docker.
To do so, we are using Consul (similar to ETCD/Confd). Consul has a DNS interface. I need to be able to use it to query the connection information of a CDAP application, which is something like "cdap_IP:port/some_stream_path.
So I want to register in consul
somedocker.whatever.domain.com as dockerhost:ip
somecdap.whatever.domain.com as cdap_IP:port/some_stream_path/streamname
And from inside of the docker I want to be able to call Consul's service discovery using DNS, e.g. Just looking up somecdap.whatever.domain.com should return to me cdap_IP:port/some_stream_path/streamname. Then the docker can post into the stream.
Except your non-allowing me to use somecdap.whatever.domain.com as an app name makes this process onerous and I have to map what I want as a name to what your API can tolerate as a name.