> It’s [sql] just not built for the complex relationships that exist in a massive organizations like NASA.
This is an absurd claim.
> Extracted Skills from Team Resumes
> Extracted Skills
> Subject Matter Experts Finder Question: designed to identify employees with expertise in specific domains or mission-critical capabilities.
I can't think of anything that screams "incompetent management" more than this. So, to find a subject matter expert, you're going to "extract skills" and "extract resumes" to answer abstract questions about your staff... without ever once.. just _talking_ to your staff?
What a cold and bizarre future these people think we want to live in.
Meanwhile can we use technology to improve the level of connectivity I have and experience as an employee? Can you please stop asking LLMs to "extract" things about me into goofy automated pipelines? If you want skilled workers then you need to demonstrate skilled management. This is all the exact opposite of that.
It doesn't seem like it is intended for a manager to use to get information about their direct reports, but rather org-wide information?
NASA apparently has ~18k employees, it seems like it might be useful to be able to query "Who at NASA has X, Y, Z skills that can help us with this project." Then you can speak to some of those people face-to-face. It won't be perfect but certainly sounds like a useful tool in principle.
This is like saying: "Here we have a rocket, but let's keep trying to go to the moon by bike." xD
What's wrong with attempting to better understand a given organization using LLMs or any other tech? Ofc, great managers will try as hard as possible to talk face to face as much as possible.
I highly doubt the difference between current staff management and adding this thin layer is equivalent to difference between a bike and a rocket. It's more like saying "we get to the moon just fine, but if we strap this extra booster on, we will get there 2% faster than before but with all kinds of additional risks to the payload!"
> What's wrong with attempting to better understand a given organization
You can alienate your employees and lose your skill base as a result. I'd like to be evaluated based on upon my work and dedication, not what some LLM thinks it sees in my resume. I've worked for my current company for 17 years. My resume contains none of that work or any skills gained in that time.
I also like to take on new challenges and learn new skills. The LLMs "extractions" cannot see this or attend to it.
> Ofc, great managers will try as hard as possible to talk face to face as much as possible.
That's not the problem being discussed here. The question is "can we use technology to make better organizational decisions particularly when it comes to the efficient use of human resources." If I have a bad boss, I'm going to quit, and you'll never even have this opportunity. If I have a good boss, and you interfere with his decisions using LLM driven logic, I'm going to quit, and you're never going to get the benefit of that labor anyways.
My consulting org has 3 (three) right now running in parallel and had another few over the last decade. Yet people still send emails around asking for support. And as a consultant every one is mandatory and every one is a bad joke in its own way. I don't think the market is anything else than a huge scam. Is this tool here better? If yes, hats off.
DISCLAIMER: The co-founder and CTO of Memgraph here.
To add more context, Memgraph Enterprise pricing is explained under https://memgraph.com/pricing: "Starting at $25,000 per year for 16 GB, Memgraph has an all-inclusive, simple pricing model that scales with your workload without restrictions. No charge for compute. No charge for replicas. No charge for algorithms. No Surprises.".
In addition, Memgraph Community is free (standard BSL license, which turns into Apache2 4 years after release date, https://github.com/memgraph/memgraph/blob/master/licenses/BS...), and it has many features that are usually considered enterprise (users, replication, not a single degradation in performance or scale, etc.).
Please elaborate more about why the pricing seems expensive, or put it into the infra-cost perspective :pray:
I think on this site anything that's more expensive than free is considered expensive. Countless arguments have been had on Oracle vs Postgres, including lock-in. I think lock-in is more important to consider than license cost.
To be fair, it is quite nice for the pricing to be transparent. And I think it's somewhat competitive w.r.t. Stardog, for example. The community version is less restricted than Ontotext, for example.
If you want a production-grade graph DBMS, you don't have that many OSS options that are reliable and well-supported.
In the relational space, it took OSS options like Postgres many decades (and somehow paid-for person-years) to get to a place where enterprises seriously consider migrating off Oracle to it.
A Configuration Management Database (CMDB) is a centralized repository that stores information about Configuration Items (CIs), including their attributes and relationships. It's a key component of IT service management (ITSM), providing visibility into the components that make up IT services, like hardware, software, and documentation
It can be incredibly useful. One example is to have every every process linked to the VM it is running on and the host the VM is running on and the TCP port the process is listening on. If you have all the correct relationships defined then you can write a query like this to find every process on every VM listening on port 80 on a given VM host.
MATCH p = (host:VMHOST {name: 'your_host_name'})-[:RUNS]->(vm)-[:HAS_SERVICE]->(service)-[:EXPOSES_PORT]->(port:TCPPORT {port: 80})
RETURN p
This can save a absurd amount of time for analyzing the impacts of failures and security isolation compliance.
Not very big since it is only used internally at my company. 5 digit node count and high 6 digit relationships count. Queries are usually very fast unless you try to do something stupid that ends up having to search the entire graph. indexing critical high-cardinality properties and thinking of relationships as a kind of index help a lot with query performance. I have been meaning to test how fast memgraph is.
This is such an overkill for that kind of data. Even if they do plan to "scale up significantly", I doubt that they'll actually experience any benefit of graph db.
Man, love seeing pushback on automated skill matchingsometimes feels like tech folks keep inventing new tools just to dodge actual conversations. Ever wonder if all this automation just makes things colder instead of smarter?
> "To make sure everyone understands that, I prefer label property graphs over RDF."
I have two major issues with virtually all graph DBMSs that are not RDF/SPARQL-based:
1) They do not allow structure-preserving querying. That is, I query a graph and want the results to be a smaller graph. This is trivial in SQL, you just 'SELECT * FROM x WHERE ...' and the result set you get is tabular just like the table x. In SPARQL, there are a CONSTRUCT/DESCRIBE queries that do just that - give you the results as a graph.
2) They don't use any (internationally recognized) standard to represent graph data. RDF is the only such format known to me (ignore all the semantic web stuff associated with it and just consider the format).
230k edges is peanuts for a graph db. It's like when the number of rows times columns in your SQL DB is 230k. NASA could (should?) have just used Oxigraph, RDF4J, or Jena. Stardog and Ontotext are the paid options. However, it is quite nice to see more interest in graph-based DBMSs in general!
> “Which employees have cross-disciplinary expertise in AI/ML?”
Regarding the study itself, I did not understand who is the target user of this. I would rather be more interested in the Lessons Learned 2.0 study (I understand it was attempted once before [1]). I don't think the study at hand would be able to correctly answer questions about expertise.
On the technical side, as far as I understand, the cosine similarity was computed per triplet? In that case, I could see how pgvector could be used for this. Relevance expansion is the only thing in the article that made me think that it would be cool if it works well. But I could see how in a combo of a regular RDF DBMS + pgvector, one could first do a cosine similarity query via pgvector and then compute an (S)CBD [2] of the subject (the from node) of the triplet.
"They do not allow structure-preserving querying. That is, I query a graph and want the results to be a smaller graph."
I'm not sure what you mean by this. The result of a query in neo4j is a set of nodes with specified relations linking them. It is much more flexible than the way SQL can only return a single table.
"In the RETURN part of your query, you define which parts of the pattern you are interested in. It can
be nodes, relationships, or properties on these"
you can return all nodes, relationships, and paths that match a query by using this syntax
MATCH p = (a {name: 'A'})-[r]->(b)
RETURN *
This is the exact opposite of a rectangular result set.
As an alternative to a pure graph db (e.g. here, memgraph), has anyone here used Apache's AGE graph-database extension for Postgresql? For making a knowledge graph that can live alongside SQL?
I believe AGE has unfortunately been defunded: https://github.com/apache/age/discussions/2150
It’s a shame because it seemed like being able to query data across multiple paradigms would be really useful
NASA announced LLMs in early days (years ago) - it seemed like they wanted to understand their own document libraries! What else could be inferred here? mass layoffs plus "people substitutes" ? is there a more diplomatic way to see this?
The only connection between "they wanted to understand their own document libraries" and "mass layoffs" is potentially "increased efficiency leads to needing less people for the same job". If there's anything else, please let me know.
And if it's that, then are you suggesting to not implement a certain technological efficiency tool in order to keep (now clearly redundant) jobs? That has never worked long-term in the history of mankind, AFAIK.
Even paying a college grad to babysit a server costs more than their yearly rate. I assume you're speaking as someone who loves to host everything for themselves, but the logic is surely different in enterprise/government, no?
It depends on your usage models, but if you compare it to AWS Neptune, the pricing seems quite high. I doubt NASA is running queries 24x7 for this use case so other options could be less expensive.
Seems like a very simple use-case given that it will be barely used at scale. A few thousand employee entries and read qps a few 10s? What’s so special about it to post
My experience with tools like this is that they have only one single outcome. Piling work onto the most talented or desperate(ie need money or visa) people until they leave the org/company. Eventually leading to total skill erosion and a very low average skill/productivity across the company as people leave or hide their abilities.
Why? because there is never a reward attached. Oh you want to make me the AI resource for the agency but not remove former duties or increase my pay? Ummmm no thanks. Also things tend to happen in waves ie "AI" so everyone needs a lot from a very few people at the same time. No one ever asks how those people can be empowered. Just how can we put the screws to them so they work harder.
HR and Mgmt can f-off with their "skill resource bank" or whatever nonsense they call it this year. My skills are what I was hired for on the job description. If you want to discuss a new position or higher pay for different skills I'm very happy to talk about how I can work with the org to make that happen. Thats never the case though.
> What made you choose memgraph? ... And then Memgraph showed me the cost. That kind of sold me for time for us to be able to do that.
It's an ad post about memgraph.
Yes, domain is memgraph and it seems to be a marketing case study.
The use case at NASA isn’t even new. We built this precise thing in 2008. All standards-based.
See https://www.w3.org/2001/sw/sweo/public/UseCases/Nasa/ for a public case study.
This work led to Stardog.
Which is used extensively in NASA today—
https://gpdisonline.com/wp-content/uploads/2019/09/StardogNA...
https://www.informationweek.com/machine-learning-ai/stardog-...
> It’s [sql] just not built for the complex relationships that exist in a massive organizations like NASA.
This is an absurd claim.
> Extracted Skills from Team Resumes
> Extracted Skills
> Subject Matter Experts Finder Question: designed to identify employees with expertise in specific domains or mission-critical capabilities.
I can't think of anything that screams "incompetent management" more than this. So, to find a subject matter expert, you're going to "extract skills" and "extract resumes" to answer abstract questions about your staff... without ever once.. just _talking_ to your staff?
What a cold and bizarre future these people think we want to live in.
Meanwhile can we use technology to improve the level of connectivity I have and experience as an employee? Can you please stop asking LLMs to "extract" things about me into goofy automated pipelines? If you want skilled workers then you need to demonstrate skilled management. This is all the exact opposite of that.
It doesn't seem like it is intended for a manager to use to get information about their direct reports, but rather org-wide information?
NASA apparently has ~18k employees, it seems like it might be useful to be able to query "Who at NASA has X, Y, Z skills that can help us with this project." Then you can speak to some of those people face-to-face. It won't be perfect but certainly sounds like a useful tool in principle.
This is like saying: "Here we have a rocket, but let's keep trying to go to the moon by bike." xD
What's wrong with attempting to better understand a given organization using LLMs or any other tech? Ofc, great managers will try as hard as possible to talk face to face as much as possible.
> This is like saying:
I highly doubt the difference between current staff management and adding this thin layer is equivalent to difference between a bike and a rocket. It's more like saying "we get to the moon just fine, but if we strap this extra booster on, we will get there 2% faster than before but with all kinds of additional risks to the payload!"
> What's wrong with attempting to better understand a given organization
You can alienate your employees and lose your skill base as a result. I'd like to be evaluated based on upon my work and dedication, not what some LLM thinks it sees in my resume. I've worked for my current company for 17 years. My resume contains none of that work or any skills gained in that time.
I also like to take on new challenges and learn new skills. The LLMs "extractions" cannot see this or attend to it.
> Ofc, great managers will try as hard as possible to talk face to face as much as possible.
That's not the problem being discussed here. The question is "can we use technology to make better organizational decisions particularly when it comes to the efficient use of human resources." If I have a bad boss, I'm going to quit, and you'll never even have this opportunity. If I have a good boss, and you interfere with his decisions using LLM driven logic, I'm going to quit, and you're never going to get the benefit of that labor anyways.
Every large consulting org does this. It’s a market for enterprise and home-grown solutions.
My consulting org has 3 (three) right now running in parallel and had another few over the last decade. Yet people still send emails around asking for support. And as a consultant every one is mandatory and every one is a bad joke in its own way. I don't think the market is anything else than a huge scam. Is this tool here better? If yes, hats off.
I know it’s a marketing case study, but:
> Ever wondered how NASA identifies its top experts, forms high-performing teams, and plans for the skills of tomorrow?
Here’s another resource on that https://appel.nasa.gov/2010/02/18/aa_2-7_f_nasa_teams-html/ the book “How NASA Builds Teams: Mission Critical Soft Skills for Scientists, Engineers, and Project Teams”
Memgraph is laughably expensive - I honestly wonder what anyone actually uses it for outside of companies that just don't care about infra spend.
DISCLAIMER: The co-founder and CTO of Memgraph here.
To add more context, Memgraph Enterprise pricing is explained under https://memgraph.com/pricing: "Starting at $25,000 per year for 16 GB, Memgraph has an all-inclusive, simple pricing model that scales with your workload without restrictions. No charge for compute. No charge for replicas. No charge for algorithms. No Surprises.".
In addition, Memgraph Community is free (standard BSL license, which turns into Apache2 4 years after release date, https://github.com/memgraph/memgraph/blob/master/licenses/BS...), and it has many features that are usually considered enterprise (users, replication, not a single degradation in performance or scale, etc.).
Please elaborate more about why the pricing seems expensive, or put it into the infra-cost perspective :pray:
I think on this site anything that's more expensive than free is considered expensive. Countless arguments have been had on Oracle vs Postgres, including lock-in. I think lock-in is more important to consider than license cost.
To be fair, it is quite nice for the pricing to be transparent. And I think it's somewhat competitive w.r.t. Stardog, for example. The community version is less restricted than Ontotext, for example.
Not really competitive with Stardog given our leading LLM integration with Voicebox. 85% pass@1 to exit POV with new customer.
If you want a production-grade graph DBMS, you don't have that many OSS options that are reliable and well-supported.
In the relational space, it took OSS options like Postgres many decades (and somehow paid-for person-years) to get to a place where enterprises seriously consider migrating off Oracle to it.
Are there any? My experience so far with graph databases is a resounding failure.
I'm using Neo4j to build a CMDB and it is awesome.
What is a "CMDB"?
A Configuration Management Database (CMDB) is a centralized repository that stores information about Configuration Items (CIs), including their attributes and relationships. It's a key component of IT service management (ITSM), providing visibility into the components that make up IT services, like hardware, software, and documentation
Interesting!
It can be incredibly useful. One example is to have every every process linked to the VM it is running on and the host the VM is running on and the TCP port the process is listening on. If you have all the correct relationships defined then you can write a query like this to find every process on every VM listening on port 80 on a given VM host.
MATCH p = (host:VMHOST {name: 'your_host_name'})-[:RUNS]->(vm)-[:HAS_SERVICE]->(service)-[:EXPOSES_PORT]->(port:TCPPORT {port: 80}) RETURN p
This can save a absurd amount of time for analyzing the impacts of failures and security isolation compliance.
That's good to hear, how large is the graph you're building (nodes, edges) and how do queries perform?
The big issue we have had with Neo4J was with replication, when we do MASSIVE updates. For the rest, it handles the charge reasonably well.
What constitutes a massive update?
Not very big since it is only used internally at my company. 5 digit node count and high 6 digit relationships count. Queries are usually very fast unless you try to do something stupid that ends up having to search the entire graph. indexing critical high-cardinality properties and thinking of relationships as a kind of index help a lot with query performance. I have been meaning to test how fast memgraph is.
In OSS or generally?
Either, tbh?
> The current graph has about 27K nodes and 230K edges
That is tiny even by historical standards. I was expecting there to be some type of technology here. Why is this interesting?
> 27K nodes and 230K edges
This is such an overkill for that kind of data. Even if they do plan to "scale up significantly", I doubt that they'll actually experience any benefit of graph db.
Why do you say that?
Man, love seeing pushback on automated skill matchingsometimes feels like tech folks keep inventing new tools just to dodge actual conversations. Ever wonder if all this automation just makes things colder instead of smarter?
And slower, if we‘re honest, since they never solve issues they intend to solve 100% and human attention is still needed (which is good)
> "To make sure everyone understands that, I prefer label property graphs over RDF."
I have two major issues with virtually all graph DBMSs that are not RDF/SPARQL-based:
1) They do not allow structure-preserving querying. That is, I query a graph and want the results to be a smaller graph. This is trivial in SQL, you just 'SELECT * FROM x WHERE ...' and the result set you get is tabular just like the table x. In SPARQL, there are a CONSTRUCT/DESCRIBE queries that do just that - give you the results as a graph.
2) They don't use any (internationally recognized) standard to represent graph data. RDF is the only such format known to me (ignore all the semantic web stuff associated with it and just consider the format).
230k edges is peanuts for a graph db. It's like when the number of rows times columns in your SQL DB is 230k. NASA could (should?) have just used Oxigraph, RDF4J, or Jena. Stardog and Ontotext are the paid options. However, it is quite nice to see more interest in graph-based DBMSs in general!
> “Which employees have cross-disciplinary expertise in AI/ML?”
Regarding the study itself, I did not understand who is the target user of this. I would rather be more interested in the Lessons Learned 2.0 study (I understand it was attempted once before [1]). I don't think the study at hand would be able to correctly answer questions about expertise.
On the technical side, as far as I understand, the cosine similarity was computed per triplet? In that case, I could see how pgvector could be used for this. Relevance expansion is the only thing in the article that made me think that it would be cool if it works well. But I could see how in a combo of a regular RDF DBMS + pgvector, one could first do a cosine similarity query via pgvector and then compute an (S)CBD [2] of the subject (the from node) of the triplet.
[1]: https://youtu.be/QEBVoultYJg?t=1653
[2]: https://patterns.dataincubator.org/book/bounded-description....
"They do not allow structure-preserving querying. That is, I query a graph and want the results to be a smaller graph."
I'm not sure what you mean by this. The result of a query in neo4j is a set of nodes with specified relations linking them. It is much more flexible than the way SQL can only return a single table.
Query result in openCypher is a similar rectangular result set like in SQL. See openCypher spec p. 74.
"In the RETURN part of your query, you define which parts of the pattern you are interested in. It can be nodes, relationships, or properties on these"
you can return all nodes, relationships, and paths that match a query by using this syntax
MATCH p = (a {name: 'A'})-[r]->(b) RETURN *
This is the exact opposite of a rectangular result set.
As an alternative to a pure graph db (e.g. here, memgraph), has anyone here used Apache's AGE graph-database extension for Postgresql? For making a knowledge graph that can live alongside SQL?
I believe AGE has unfortunately been defunded: https://github.com/apache/age/discussions/2150 It’s a shame because it seemed like being able to query data across multiple paradigms would be really useful
My dream databases is Neo4j style relationships and MongoDB style documents.
Databases like that do exist. There is ArangoDB for example. I have no idea if it’s any good
Check out SurrealDB. It might be exactly what you're looking for. https://surrealdb.com/
HP Tried this over 20 years ago. It stranded in HR and union disputes.
My own take is this will just be gamed to the max by ladder climbers looking out for number 1.
NASA announced LLMs in early days (years ago) - it seemed like they wanted to understand their own document libraries! What else could be inferred here? mass layoffs plus "people substitutes" ? is there a more diplomatic way to see this?
The only connection between "they wanted to understand their own document libraries" and "mass layoffs" is potentially "increased efficiency leads to needing less people for the same job". If there's anything else, please let me know.
And if it's that, then are you suggesting to not implement a certain technological efficiency tool in order to keep (now clearly redundant) jobs? That has never worked long-term in the history of mankind, AFAIK.
I think I found a place Dodge can save some money. Memgraph pricing is ridiculous.
Even paying a college grad to babysit a server costs more than their yearly rate. I assume you're speaking as someone who loves to host everything for themselves, but the logic is surely different in enterprise/government, no?
It depends on your usage models, but if you compare it to AWS Neptune, the pricing seems quite high. I doubt NASA is running queries 24x7 for this use case so other options could be less expensive.
Seems like a very simple use-case given that it will be barely used at scale. A few thousand employee entries and read qps a few 10s? What’s so special about it to post
My experience with tools like this is that they have only one single outcome. Piling work onto the most talented or desperate(ie need money or visa) people until they leave the org/company. Eventually leading to total skill erosion and a very low average skill/productivity across the company as people leave or hide their abilities.
Why? because there is never a reward attached. Oh you want to make me the AI resource for the agency but not remove former duties or increase my pay? Ummmm no thanks. Also things tend to happen in waves ie "AI" so everyone needs a lot from a very few people at the same time. No one ever asks how those people can be empowered. Just how can we put the screws to them so they work harder.
HR and Mgmt can f-off with their "skill resource bank" or whatever nonsense they call it this year. My skills are what I was hired for on the job description. If you want to discuss a new position or higher pay for different skills I'm very happy to talk about how I can work with the org to make that happen. Thats never the case though.
Talks extensively about the details of the thing.
But doesn't actually show the thing.
That's AI hypecycle signal for probably bullshit/defective thing.
Yall know that Apache Age is a thing to run Cypher in Postgres right?
Interesting, i am just going to plug something I just built around this concept: https://skillriskaudit.com/
Would love for some feedback!
Any idea which LLM they're using?